Computational Creativity
Instructor Notes
Douglas Fisher
Note: These notes are to be used in conjunction with the slides for the course. Perhaps with a few exceptions there are no figures or examples in these notes – these features are in the slides. Both these notes and the slides are arranged by week and it should be easy to associate material between illustrations in the slides with descriptive text in these notes. If not, please ask. For weekly overviews see the Schedule at the end of the Syllabus.
Week 1
This course explores means by which artificially intelligent systems exhibit behaviors that can be regarded as creative, in areas such as visual arts, music, storytelling, architecture, product design, and computer programming. Typically, such systems are best viewed as tools for humans to advance human creativity, or to advance a hybrid human-machine creativity, but we will touch on creativity by large collectives of agents and by autonomous AIs too.
Characteristics of creativity
Creativity, of various types and degrees, characterizes processes of creation. Creativity is a characterization in a multidimensional space relating to the extent to which a process (as embodied by some kind of agent) explores alternatives through thought and action, evaluates those alternatives, imagines what are possible results of actions, and empathizes and otherwise mentalizes the goals, capabilities, and feelings of other agents. We will not treat creativity as binary valued, but we will implicitly reduce the multiple dimensions of creativity into a single dimension to talk about creation processes as more or less creative.
The results of creativity can be objects or thoughts, and the results are often characterized by varying kinds and degrees of novelty and value – how distinct is a result relative to a collection of other objects and imaginings of similar function, and what is the result’s pragmatic utility, beauty, or interestingness. You will sometimes hear inanimate objects referred to as creative, like “that is a creative vase”. You can assume that this is shorthand for saying that the vase has high novelty or value; or that the process that created that vase did significant exploration or used novel evaluation metrics, etc. If we call an agent creative it suggests that the agent, a human or AI artist or engineer, often carries out creative processes of creation, or produces results that have high degrees of novelty and value. (If its not yet clear, whenever I use ‘or’ its probably an inclusive ‘or’.)
Is it possible for a creative process to produce mundane results? Sure, in large part because creativity lies in the eye of the beholder as well as the creator, and the space of objects against which the spectator evaluates a creation in terms of novelty and value is perhaps wider than the creator’s experience. The creator may have carried out processes that were quite unique to them, but not to a larger society. This distinction has been referred to as individual creativity and historic creativity, and of course it generalizes to a continuum between the individual and society as extremes.
Creativity occurs at different scales of time and activity, as well as of agency. It occurs in the operations at an assembly line station, however small we think of it (remember, creativity isn’t a binary), and at the level of the entire assembly line or some alternative paradigm for holistic construction. Non-human animals are creative, both as individuals and collectives, as are AIs, or so that is the assumption of this class. The topics mentioned here – exploration (search) vs exploitation, evaluation using measures such as novelty and value, projection or imagination, and scales of agency, time, and space are all amenable to computational treatment.
Computational Creativity and AI
Computational creativity is an area of artificial intelligence (AI), though I would say understudied. Most AI systems today, including computational creativity systems, are very focused and narrow applications, such as creation of visual art from narrative prompts, story writing, and music composition. In principle, such functionality in human affairs are very broad, drawing from the human creators’ wealth of experience, but it could be said that the AI systems that mimic the human artists take “shortcuts” in the creative steps, extracting features of the end results of creative processing and using these features in the creations of the AI. Nonetheless, the processing of computational creativity systems is impressive, typically in its processing of massive data sets that are typically the results of human creation. Some systems are multi-modal (e.g., taking narrative prompts as input and producing visual art, or vice versa from images to captions), which places them a step towards systems-level AIs. Systems level AIs include “intelligent” vehicles, Watson, and home assistants like Alexa. Further up the dimension of autonomy are person-comparable AIs, and these are still the stuff of science fiction.
One reason that I ask you to interrogate your own thinking as you work on creativity and computational creativity projects is so that you will develop insights into the component processes that you use and coordinate in completing each project, and tasks like them in other parts of your life. Each of the component processes may themselves be decomposed further, and implemented by existing AI tools, at least in part. The hope of course is that you will imagine ways of implementing the sub-processes and complete process computationally so that you can be leaders in the wave of systems-level and person-level creative AIs that will probably come.
Our assumption is that creativity pervades intelligent processes of all kinds in human systems, and we will be discussing ways that it might pervade AI systems too. As an aside, learning, like creativity, is also a capability that pervades processes in human systems, and should in AI systems too, but even now (machine learning has been explored since the 1950s), most machine learning lies on the narrow end of the generality dimension. Cognitive architectures and cognitive systems are areas of AI that study the processes of system-level AIs. We will often structure our discussions of creativity in systems-level AIs by considering the requisite capabilities outlined in our cognitive architectures reading.
Curation
I interpret ‘curation’ broadly, as concerning collections of many types of things by novices and experts alike. Curation includes component activities of selecting, displaying, and educating about collections. Curation is an act of creation, as in creating an exhibit, and it begs some degree of creativity. Because curation involves collections of discrete things it is a good class of activity for introducing the utility of computational approaches. Our goal is to not simply talk about existing computational tools for carrying out curation, if any, but to imagine how computational tools might be applied to curation subtasks. This theme of discussing “what is” and “what could be” will run through other topics as well, though I am less likely to know how to imagine the “what is possible” with respect to other media, though you may know more and are encouraged to speak up in class or on Piazza.
Suppose that a building for curating artwork has already been created, like the West Building of the National Gallery of Art, and a collection of paintings and sculpture is on site. A curator knows the dimensions of rooms, heights of walls, measurements of paintings and sculptures, and lots of metadata like artist, year, region, etc. What are the ways to organize the collection? Can the same methods and metrics be used to display gardens, zoos, natural histories, etc?
A basic AI functionality that might be useful for a computational curator is state-space search, where the states can be vertices in an explicit graph like a road network, or the states can be nodes or vertices (aka states) that are generated on-demand from an implicit graph. The “vertices” or states in AI problems are most typically quite complicated knowledge structures that are part of an implicit graph. State-space search is most typically used to explore alternatives in the ‘problem-solving and planning‘ and ‘reasoning and belief maintenance‘ functionality classes of a cognitive architecture.
Another basic AI strategy is representation and learning with artificial neural networks (ANNs), which are typically huge collections of simple interconnected processing units (akin to individual neurons). Powerful computation stems from the interconnection of these units and their coordinated activity. In our first treatment of ANNs the functionality that we discuss is most appropriate to three cognitive architectures functionality classes: ‘recognition and categorization‘ , ‘perception and situation assessment‘, and ‘prediction and monitoring‘.
Creativity in Language
Language evolves and creativity on the part of individuals and groups is manifest in this evolution. Lets take a look at the “semantic broadening” of the words curate’ and ‘curator’ from the week 1 readings.
According to the reading ‘curator’ appeared in the 14th century as “a person corresponding nearly to the guardian of <Roman> law and appointed to manage the affairs of a person past the age of puberty while he is a minor or of any such person when legally incompetent (as a spendthrift or a lunatic).” In the same time period it was also a synonym for ‘curate’, a member of the clergy, which upon further exploration is “a person who is invested with the care or cure (cura) of souls of a parish” according to Wikipedia. From this focus on collecting and caring for laws, persons, and souls, there was a semantic broadening in the 17th century to a person who cares for museum collections.
Curate in a verb form made its appearance in the late 19th century, as the action of “carefully choosing” and caring for collections of objects generally, where the example of libraries is given. The semantic broadening about the kind of objects implied by curate grew further in the 20th and 21st century, with examples of recipes and menus.
What I take from this history is that ‘curation’ seems to have broadened from parishes to museums to libraries, which all convey some special purposes for education and enrichment, as well as connoting some physical focal point. Perhaps the more recent broadening of meaning to allow for curation’s role at smaller scales, even by individuals, reflects a change in culture, at least in the West and the US. I was reminded of “curation can become a deeply personal expression of artistic taste” in yet another reading for this week. People are certainly passionate about cooking and food, so recipes and menus may not seem strange, but perhaps even T-shirts can be a target of curation, even if they don’t appear in museums of popular culture. Its interesting that when I read “it is gauche to apply <curated> to one’s collection of ironic t-shirts” that I misread “irony” as “iconic”. Because (I think) that I was already starting to think about the future use of the words, I thought that the statement might be shortsighted. Iconic is “regarded as a representative symbol or as worthy of veneration” (Oxford), or “distinctive excellence” (MW). So perhaps I would not scoff at the use of curation as describing selection and display of iconic materials, even T shirts by individuals, but only if it was a passion to the curator.
Perhaps we could computationally model the creativity in the evolution of word use by searching through the references in word meanings that link those meanings (e.g., curate <–> clergy<–> care
<–> souls <–> treasure
<–> paintings
<–> books
An implementation for such a computational model of semantic broadening might involve spreading activation with semantic networks, perhaps in conjunction with other computational approaches like ANNs.
Week 2
Announced on Piazza when its posted