Two researchers at the University of Texas Austin working on the DALL·E 2 program have come to a stunning conclusion: The artificial intelligence (AI) program is now inventing its own language.
DALL·E 2 is a new research initiative that uses Open AI, an artificial intelligence language launched by the research laboratory under the same name, founded by Elon Musk, Sam Altman, and Ilya Sutskever. It can create photo-realistic images and art from scratch, like this one of a koala dunking a basketball, by following simple text instructions.
DALL·E 2 can also realistically edit and retouch photos based on a simple natural language description, and it can fill in or replace part of an image with AI-generated imagery that blends seamlessly with the original. It’s called in-painting and was achieved by training a neural network on images and their text descriptions.
DALL·E 2 research has three main objectives:
- It can help people express themselves visually in ways they may not have been able to before.
- AI-generated images can tell us a lot about whether the system understands us or is just repeating what it’s been taught.
- It helps humans understand how AI systems see and understand our world.
Computer Science Ph.D. research student Giannis Daras wrote in a viral tweet last week how DALL·E 2 has developed a secret language and is inventing its own words. For instance, asking the algorithm to generate an image of “farmers talking about vegetables” generates text with the unknown word “vicootes.” But if you put the word “vicootes” into the algorithm, it spits out pictures of vegetables.
Giannis Daras and Alexandros G. Dimakis have now released a short UT research paper summarizing their findings called: Discovering the Hidden Vocabulary of DALL·E 2. In their Abstract, it reads: “We discover that DALL·E 2 seems to have a hidden vocabulary that can be used to generate images with absurd prompts. For example, it seems that Apoploe vesrreaitais means birds and Contarra ccetnxniams luryca tanniounons (sometimes) means bugs or pests. We find that these prompts are often consistent in isolation but also sometimes in combinations. We present our black-box method to discover words that seem random but have some correspondence to visual concepts. This creates important security and interpretability challenges.”
It’s important to note that the conclusions drawn by Daras are still somewhat tenuous, and the paper admits the results are not 100% consistent. Sometimes, the prompt “Contarra ccetnxniams luryca tanniounons” generates pictures of bugs, while other times, it generates images of “mostly animals.”
Research analyst Benjamin Hilton responded on Twitter. “It’s random chance. Or just maybe (if you really press me) ‘Apoploe vesrreaitais’ looks like a binomial name for some birds or bugs.”
“To me, this is all starting to look a lot more like stochastic, random noise, than a secret DALL-E language,” Hilton added.
 program is now inventing its own language. ){kind=link}