Artificial Design: Creation Versus Machine Learning
Contexte
Anthony Masure, Artificial Design: Creation Versus Machine Learning, Geneva, HEAD – Publishing, coll. “Manifestes,” 2023
Résumé
In the early 2020s, driven by progress in “artificial intelligence”, programs such as GPT-3, DALL E, Midjourney or Disco Diffusion made it possible to generate images from text instructions. Although these productions often only imitate existing data and lack diversity, media debates tend to focus on the replacement of humans by the machine. In doing so, they screen the essential question: what is the spectrum of current and potential implications of machine learning for design practices?
—
Formats
This book is available in several open access digital formats, all released under a CC BY-SA free license: PDF, HTML, ePub.
—
Associates research project
The essay Artificial Design: Creation Versus Machine Learning was written as part of a research project entitled “Design and machine learning: automation takes command?.” Funded by HES-SO and led by Anthony Masure at HEAD – Genève in 2022-2023 with Alexia Mathieu, Douglas Edric Stanley, and Élise Gay & Kévin Donnot, this project aims to question the mutations of graphic design in contact with AI.
The objective is twofold:
– Understand the current uses (projects, opportunities, etc.) and problems (blockages, lacks, etc.) of machine learning technologies for graphic designers.
– On the basis of these analyses, initiate a “projet-based research” to set up new usages adapted to independent design studios.
Several additional deliverables were produced as part of the research project, the details of which are given below.
—
Interviews with designers and artists
15 interviews with artists and graphic design studios and European pioneers in the use of machine learning were produced by Alexia Mathieu (head of Master Media Design at HEAD – Geneva): Boyd Rotgans, Cristobal Valenzuela, Deniz Kurt, Feileacan McCormick, Kevin Donnot, Marta Revuelta, Martin Tricaud , Meredith Thomas, Nadia Piet, Nicolas Barradeau, Rifke Sadleir, Simone Rebaudengo, Superposition Studio, Etienne Mineur.
These interviews shed light on emerging practices, which approach neural networks as material for creation and play partners. Extracts from the interviews are reproduced in the book Artificial Design, and the complete versions are available for consultation on the project website.
—
A website using machine learning
The designer Douglas Edric Stanley (Teacher in Master Media Design at HEAD – Geneva) has created an experimental site that includes all the texts (interviews and essay) of the project research and remixes them through text and image generators.
The text generator is Davinci, namely a Large-Language Model (LLM), the same program model behind ChatGPT. The image generator is a Generative Pre-trained Transformer, marketed under the name DALL E. We also experimented with the GPT-2, GPT-NeoX., Bloom, Disco Diffusion, Stable Diffusion, and Midjourney models.
The online HTML version of the Design sous artifice essay is “augmented” by this experimental interface: some sentences have been selected and have been re-interpreted or even “completed” by the LLM Davinci. The keywords of these texts are also transformed into images by DALL·E. In addition to these “pre-calculated” excerpts, one can also select any part of the essay to ask the generator to complete the reflection in “real time” via the use of the API (paid) from OpenAI.
Regarding the generation of images, a particular strategy had to be invented to create convincing results. It is not a simple repetition of the words underlined in the essay or the interviews. The art of “prompt engineering” is a creative work in itself and requires thoughtful strategies to give visually convincing results. A sentence copied from the site and sent as is to DALL·E will not give good results.
A pre-processing algorithm has been created, which:
1 – Detects the selected language
2 – Extract names and keywords from the selected quote
3 – Build a new “prompt” from a bank of model sentences
4 – Integrate the keywords into the model sentence
As in many AI projects, this solution is a mix of old techniques of “natural language processing” and new approaches of neural networks.
The design of the site is inspired by Smalltalk-76, the first interfaces of a complete desktop publishing (DTP) system. This interface would later inspire the Macintosh interface (1984) and serve as a model for the “personal computer” interface we still use today.
Some references used to design this interface:
– Smalltalk, Wikipedia
– Smalltalk-78 Simulator
– Goldberg Adele Kay Alan eds., Smalltalk-72 Instruction Manual, Xerox Palo Alto Research Center , March 1976
– Dan Ingalls, “Design Principles Behind Smalltalk”, Byte. McGraw-Hil, August 1981
– Smalltalk Zoo, The Computer History Museum, Mountain View , California
—
Communication campaign
On the occasion of the publication of the essay Artificial Design in March 2023, the graphic designers Élise Gay and Kévin Donnot (E+ K) were invited to produce a communication campaign in the form of a carte blanche. Their proposal, titled Davinci’s Insights, questions the relationship between graphic design and AI through a series of five posters and a short film. Beyond the communication issues, this critical project problematizes the questions developed in the book in a visual form that challenges the viewer.
Considering Davinci (an artificial intelligence trained with the GPT-3 model of OpenAI) as the third member of their studio, Élise Gay and Kévin Donnot engaged with him in a discussion on the various questions raised by human-machine collaborations in the context of graphic design. Each concept is embodied in a manifest object represented as a three-dimensional model generated by the neural network CLIP-Mesh: a Rubick’s cube, a robot equipped with an AI, a set of gears, a brain, and a book equipped with a computer chip. Far from the search for visual realism or the simulation of tangible artifacts, the style of these digital objects evokes cubist volumes with fuzzy and ambiguous materials, echoing what the Japanese roboticist Masahiro Mori called the uncanny valley.
The short film offers a long tracking shot in a fictional space where the objects are gradually constructed following the off voice declaiming the exchange, while the series of posters synthesizes each issue into an image associating the generated object, the question designers and the response of this other intelligence. This visual universe was declined in a series of posts for social media to announce the release of the trial.
E+K, Davinci’s Insights, court-métrage 3D, 2023, 6’34
—
—
Awards and exhibitions
May 2023: 1st prize from the Académie des Beaux Arts for the Davinci’s Insights project awarded to studio E+K at the Biennale de Design graphique de Chaumont (France) as part of the 30th international poster competition.
May to October 2023: Exhibition of the Davinci’s Insights project at the Biennale de Design graphique de Chaumont (France) as part of the selection for the 30th international poster competition, and in “Procès d’intention” (curated by Jean-Michel Geridan).
—
Press release
Graphic Design: Alicia Dubuis.