Live data from Hacker News

DALL-E 2 has a secret language

twitter.com

21–30 of 118 posts

Re: DALL-E 2 has a secret language

#21

Interestingly Google detects these words as Greek. I know they are nonsensical and not actually Greek but I'm wondering if any Greek speakers might be able to provide some insights. Are these gibberish words close to meaningful words? (clear shot in the dark here) Maybe a linguist could find more meaning?

Or maybe it’s a subtle joke by Google as a play on the idiom “it’s all Greek to me”?

Re: DALL-E 2 has a secret language

#22

Interestingly Google detects these words as Greek. I know they are nonsensical and not actually Greek but I'm wondering if any Greek speakers might be able to provide some insights. Are these gibberish words close to meaningful words? (clear shot in the dark here) Maybe a linguist could find more meaning?

As a native Greek, no, they don't make any sense.. sort of. My hunch is that they read significantly more like Latin than they do Greek. However it tells us something about google translate.

The reason "Apoploe vesrreaitais" is detected as Greek is because the first "word" is "phonetically" similar to the word απόπλους, which means sailing/shipping and it is rooted in ancient Greek. If we were to write Αποπλοuς using roman characters, we would write apoplous or apoloi (plural, in Greek is αποπλοΐ). So I think that the model understands that "oe" suffix is used to represent the Greek suffix "οι" that is used for plurals. The rest of the word is rather close phonetically, so there is some model that maps phonetic representations to the correct word.

The other phrase seems to be combined of words classified as Portuguese, Spanish, Lithuanian, and Luxembourgish.

Re: DALL-E 2 has a secret language

#24

Interestingly Google detects these words as Greek. I know they are nonsensical and not actually Greek but I'm wondering if any Greek speakers might be able to provide some insights. Are these gibberish words close to meaningful words? (clear shot in the dark here) Maybe a linguist could find more meaning?

One could conjecture that "Apoploe" is similar to από πουλί, "from bird". But I don't have much support for that conjecture.

The word is απόπλους, or αποπλοΐ

Re: DALL-E 2 has a secret language

#26
post #16

One of the replies is a thread with a fairly convincing rebuttal, with examples: https://twitter.com/Thomas_Woodside/status/15317102510150819...

We don't know the rules or grammar of this "language". Maybe nouns change based on how they are used

https://en.wikipedia.org/wiki/Declension

Re: DALL-E 2 has a secret language

#27
post #16

One of the replies is a thread with a fairly convincing rebuttal, with examples: https://twitter.com/Thomas_Woodside/status/15317102510150819...

A rebuttal to the rebuttal (without examples)...

How many French people speak Breton?

Re: DALL-E 2 has a secret language

#28

Interestingly Google detects these words as Greek. I know they are nonsensical and not actually Greek but I'm wondering if any Greek speakers might be able to provide some insights. Are these gibberish words close to meaningful words? (clear shot in the dark here) Maybe a linguist could find more meaning?

As a native Greek, no, they don't make any sense.. sort of. My hunch is that they read significantly more like Latin than they do Greek. However it tells us something about google translate. The reason "Apoploe vesrreaitais" is detected as Greek is because the first "word" is "phonetically" similar to the word απόπλους, which means sailing/shipping and it is rooted in ancient Greek. If we were to write Αποπλοuς using…

This is a great response (I also suspected we'd learn something from the Google Translate black box). And I agree with the idea of being closer to Latin gibberish. The phonetic relationships are a great hint to what's actually going on.

My hypothesis here is more that these models are trained more on western languages than others and thus our latent representation of "language" is going to appear like Latin gibberish due to a combination of the evolution of these languages as well as human bias. ("It's all Greek to me")

Re: DALL-E 2 has a secret language

#29
I tried a few of these in one of the available CLIP-guided diffusion notebooks, but wasn't able to get anything that looks like DALL-E meanings. Not sure if DALL-E retrained CLIP (I don't think they did?), but it maybe suggests that whatever weirdness is going on here is on the decoder side?

All the cool images that DALL-E spits out are fun to look at, but this sort of thing is an even more interesting experiment in my book. I've been patiently sitting on the waitlist for access, but I can't wait to play around with it.

Re: DALL-E 2 has a secret language

#30
post #9

Was DALL-E 2 trained on captions from multiple languages? If so, this makes a lot of sense. Somewhere early in the model the words "bird", "vogel", "oiseau" and "pájaro" have to be mapped to the same concept. And "Apoploe vesrreaitais" happens to map to the same concept. Or maybe "Apoploe vesrreaitais" is rather the tokenization of that concept, since it also appears in the output. So in a sense DALL-E is using an in…

But that's expected behavior for a language model (especially VAEs), where's the novelty? In a VAE, the vectors are probabilistic in the latent space so this is basically the NLP version of the classic VAE facial image generation where you can tweak the parameters to emphasize or de-emphasize a feature.
Post reply on HN