Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DALL-E 2 has a secret language

twitter.com

Post reply on HN
Print view View original

51–60 of 118 posts

Re: DALL-E 2 has a secret language

#51

Post by astrange »

notimpotent wrote: post #31

My first thought upon reading this: what if DALL-E (or a similar AI) uncovers some kind of hidden universal language that is somehow more "optimal" than any existing language? i.e. anything can be completely described in a more succinct manner than any current spoken language. Or maybe some kind of universal language that naturally occurs and any semi-intelligence life can understand it. Fun stuff!

That’s not possible - it’s like asking for a compression system that can compress any message.

All human languages are about the same efficiency when spoken, but of course this mainly depends on having short enough words for the most common concepts in the specific thing you’re talking about.

https://www.science.org/content/article/human-speech-may-hav...

And there can’t be a universal language because the symbols (words) used are completely arbitrary even if the grammar has universal concepts.

Re: DALL-E 2 has a secret language

#52

Post by tomrod »

melony wrote: post #30
wongarsu wrote: post #9

Was DALL-E 2 trained on captions from multiple languages? If so, this makes a lot of sense. Somewhere early in the model the words "bird", "vogel", "oiseau" and "pájaro" have to be mapped to the same concept. And "Apoploe vesrreaitais" happens to map to the same concept. Or maybe "Apoploe vesrreaitais" is rather the tokenization of that concept, since it also appears in the output. So in a sense DALL-E is using an in…

But that's expected behavior for a language model (especially VAEs), where's the novelty? In a VAE, the vectors are probabilistic in the latent space so this is basically the NLP version of the classic VAE facial image generation where you can tweak the parameters to emphasize or de-emphasize a feature.

Novel in engineering together of multiple concepts, if nothing else!

Re: DALL-E 2 has a secret language

#56

Post by ml_basics »

I find it really interesting how these new large models (DALLE, GPT3, PaLM etc) are opening up new research areas that do not require the same massive resources required to actually train the models.

This may act as a counter balance to the trends of the last few years of all major research becoming concentrated in a few tech companies.

Re: DALL-E 2 has a secret language

#58

Post by Veedrac »

Wow, I am totally going to need to wait for more experimentation before believing any given thing here, but this seems like a big deal.

It's one thing if DALL-E 2 was trying to map words in the prompt to their letter sequences and failing because of BPEs; that shows an impressive amount of compositionality but it's still image-model territory. It's another if DALL-E 2 was trying to map the prompt to semantically meaningful content and then failing to finish converting that content to language because it's too small and diffusion is a poor fit for language generation. That makes for worse images but it says terrifying things about how much DALL-E 2 has understood the semantic structure of dialog in images, and how this is likely to change with scale. Normally I'd expect the physical representation to precede semantic understanding, not follow it!

That said I reiterate that a degree of skepticism seems warranted at this point.

Re: DALL-E 2 has a secret language

#60

Post by sudosysgen »

dwallin wrote: post #36
jsnell wrote: post #16

One of the replies is a thread with a fairly convincing rebuttal, with examples: https://twitter.com/Thomas_Woodside/status/15317102510150819...

I'm not sure it's a convincing rebuttal, the examples shown all seem to have some visible commonality. Eg. "Apoploe vesrreaitais" Could refer to something along the lines of a "fan / wedge" or "wing-like" If you look at the examples of cheese, when compared to the "birds and cheese" the cheese tends to be laid out in a fan like pattern and shaped in sharp angled wedges.

It seems to refer to "bird plant" which means birds on trees, so it would make sense there would be cheese and plants if it can't find how to fit a bird.
Post reply on HN