Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DALL-E 2 has a secret language

twitter.com

Post reply on HN
Print view View original

91–100 of 118 posts

Re: DALL-E 2 has a secret language

#91

Post by voxl »

a_f wrote: post #83
jws wrote: post #5

In short: DALLE-2 generates apparent gibberish for text in some circumstances, but feeding the gibberish back in gets recognized and you can tease out the meaning of words in this unknown language.

Is it gibberish in the true sense, or is it some sort of AI generated/learned latin text for the input models used? I wonder if they used a large number of biological images in their training data along with their scientific names, which led to this second order effect.

I think calling it gibberish is a misnomer, it would be gibberish if inconsistent, but if the same strings of characters lead to the same semantic objects then that is not gibberish.

Re: DALL-E 2 has a secret language

#92

Post by IncRnd »

noizejoy wrote: post #21
godelski wrote: post #10

Interestingly Google detects these words as Greek. I know they are nonsensical and not actually Greek but I'm wondering if any Greek speakers might be able to provide some insights. Are these gibberish words close to meaningful words? (clear shot in the dark here) Maybe a linguist could find more meaning?

Or maybe it’s a subtle joke by Google as a play on the idiom “it’s all Greek to me”?

Or for something that is only somewhat subtle, it's a chicken and egg problem.

Re: DALL-E 2 has a secret language

#94

Post by PebblesRox »

codeflo wrote: post #44
TOMDM wrote: post #34

Shouldn't this be expected to a certain extent? Gibberish has to map _somewhere_ in the models concept space. Whether is maps onto anything we'd recognise as consistent doesn't mean that the AI wouldn't have some concept of where it relates, as other people have noted, the gibberish breaks down when you move it into another context, but who's to say that Dall-E 2 isn't remaining consistent to some concept it understa…

I mean, everything is easy to predict in retrospect. :) Personally, I’m a bit surprised that it has learned any connection between the letters in the generated image and the prompt text at all. I had assumed (somewhat falsely it seems) that the gibberish means that the generator just thinks of text as a “pretty pattern” that it fills in without meaning. For example, a recent post on HN suggested that it likes the wor…

Yes, specifically a prompt about Thomas Bayes generated the caption "Bay of Tayees" and the theory was that "Bayes" got corrupted to "Bay of" because of maps.

I agree that this shows a focus on the appearance of the words rather than their meaning.

https://astralcodexten.substack.com/p/a-guide-to-asking-robo...

Re: DALL-E 2 has a secret language

#95

Post by nutanc »

adastra22 wrote: post #88
nutanc wrote: post #85

I don't think it's a secret language per se. It's just that the tokens generated for these sentences are for some reason coming close to a bird latent space. Maybe if we can dig deep and do a google search for kinds of birds we can find the connection. Tokens from OpenAI below. https://t.co/Of8CBGdGAE . Found this answer: https://twitter.com/BarneyFlames/status/1531736708903051265?...

> I don't think it's a secret language per se. It's just that the tokens generated for these sentences are for some reason coming close to a bird latent space. Serious question: what else do you think language is? How else would your brain associate the word "bird" with the concept?

I would associate a bird with a bird. I wouldn't associate birmongle with bird just because both start with the token "bir". What Dalle is doing here is what it has been trained to do. Just find the closest token match and try to draw something. It does not understand something. And that's absolutely fine. I am just saying it does not have a secret language or something. It's just regular language and it's closest matches.

Re: DALL-E 2 has a secret language

#96

Post by aetherson »

nutanc wrote: post #95
adastra22 wrote: post #88

Earlier quoted context omitted.

> I don't think it's a secret language per se. It's just that the tokens generated for these sentences are for some reason coming close to a bird latent space. Serious question: what else do you think language is? How else would your brain associate the word "bird" with the concept?

I would associate a bird with a bird. I wouldn't associate birmongle with bird just because both start with the token "bir". What Dalle is doing here is what it has been trained to do. Just find the closest token match and try to draw something. It does not understand something. And that's absolutely fine. I am just saying it does not have a secret language or something. It's just regular language and it's closest ma…

I think the weird thing is that it creates these "words" and uses them in its output.

Re: DALL-E 2 has a secret language

#97

Post by adastra22 »

nutanc wrote: post #95
adastra22 wrote: post #88

Earlier quoted context omitted.

> I don't think it's a secret language per se. It's just that the tokens generated for these sentences are for some reason coming close to a bird latent space. Serious question: what else do you think language is? How else would your brain associate the word "bird" with the concept?

I would associate a bird with a bird. I wouldn't associate birmongle with bird just because both start with the token "bir". What Dalle is doing here is what it has been trained to do. Just find the closest token match and try to draw something. It does not understand something. And that's absolutely fine. I am just saying it does not have a secret language or something. It's just regular language and it's closest ma…

But there's nothing particularly avian about the phonemes "bird". Your brain just matches that pattern of sounds to the nearest concept in thought-space, which happens to describe birds. That's what language is, a consistent mapping of words to concepts.

Which seems to also be what this article is describing.

Re: DALL-E 2 has a secret language

#98

Post by totetsu »

TOMDM wrote: post #89
astrange wrote: post #86

Earlier quoted context omitted.

Prior art with GPT2: https://www.thisworddoesnotexist.com/

ok, so proposed study design, provide a sample of these along with obscure english words to a number of individuals, and get them to try pick out the real words. From there take the selection of the fake words people ranked the most real. Select a number of those words and get Dall-E 2 to try and make images of them, then see how many of those images contain results that represent the imaginary word. If anyone who ha…

Apparently you can suggest prompts to their Instagram account.

Re: DALL-E 2 has a secret language

#99

Post by afro88 »

I love the weird edge cases of ML. Imagine discussing security concerns and saying "what if it creates it's own secret language that we don't know about, which is discovered later, and people can use to circumvent privacy and obscenity controls?"

Re: DALL-E 2 has a secret language

#100

Post by civilized »

TOMDM wrote: post #34

Shouldn't this be expected to a certain extent? Gibberish has to map _somewhere_ in the models concept space. Whether is maps onto anything we'd recognise as consistent doesn't mean that the AI wouldn't have some concept of where it relates, as other people have noted, the gibberish breaks down when you move it into another context, but who's to say that Dall-E 2 isn't remaining consistent to some concept it understa…

> Gibberish has to map _somewhere_ in the models concept space.

Why? It could just go to noise images, or vaguely real-looking objects that don't look like anything in particular.

Post reply on HN