Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DALL-E 2 has a secret language

twitter.com

Post reply on HN
Print view View original

111–118 of 118 posts

Re: DALL-E 2 has a secret language

#111

Post by thaumasiotes »

TOMDM wrote: post #34

Shouldn't this be expected to a certain extent? Gibberish has to map _somewhere_ in the models concept space. Whether is maps onto anything we'd recognise as consistent doesn't mean that the AI wouldn't have some concept of where it relates, as other people have noted, the gibberish breaks down when you move it into another context, but who's to say that Dall-E 2 isn't remaining consistent to some concept it understa…

> Gibberish has to map _somewhere_ in the models concept space.

No, it doesn't. The model in use maps all input to some output, but that isn't a necessary feature of the problem at all. It's actually a terrible idea.

Re: DALL-E 2 has a secret language

#112

Post by actionfromafar »

PebblesRox wrote: post #94
codeflo wrote: post #44

Earlier quoted context omitted.

I mean, everything is easy to predict in retrospect. :) Personally, I’m a bit surprised that it has learned any connection between the letters in the generated image and the prompt text at all. I had assumed (somewhat falsely it seems) that the gibberish means that the generator just thinks of text as a “pretty pattern” that it fills in without meaning. For example, a recent post on HN suggested that it likes the wor…

Yes, specifically a prompt about Thomas Bayes generated the caption "Bay of Tayees" and the theory was that "Bayes" got corrupted to "Bay of" because of maps. I agree that this shows a focus on the appearance of the words rather than their meaning. https://astralcodexten.substack.com/p/a-guide-to-asking-robo...

I like how psychology (or at least behavioural studies) is edging closer to being relevant in computer science.

Re: DALL-E 2 has a secret language

#113

Post by rob74 »

PebblesRox wrote: post #94
codeflo wrote: post #44

Earlier quoted context omitted.

I mean, everything is easy to predict in retrospect. :) Personally, I’m a bit surprised that it has learned any connection between the letters in the generated image and the prompt text at all. I had assumed (somewhat falsely it seems) that the gibberish means that the generator just thinks of text as a “pretty pattern” that it fills in without meaning. For example, a recent post on HN suggested that it likes the wor…

Yes, specifically a prompt about Thomas Bayes generated the caption "Bay of Tayees" and the theory was that "Bayes" got corrupted to "Bay of" because of maps. I agree that this shows a focus on the appearance of the words rather than their meaning. https://astralcodexten.substack.com/p/a-guide-to-asking-robo...

In the spirit of that article, I wonder what DALL-E would spit out if you ask for "GilaWhamm" - probably images of scary medieval-looking men wielding scary medieval cutting weapons?

Re: DALL-E 2 has a secret language

#114

Post by Beltiras »

joshcryer wrote: post #43
dwallin wrote: post #36

Earlier quoted context omitted.

I'm not sure it's a convincing rebuttal, the examples shown all seem to have some visible commonality. Eg. "Apoploe vesrreaitais" Could refer to something along the lines of a "fan / wedge" or "wing-like" If you look at the examples of cheese, when compared to the "birds and cheese" the cheese tends to be laid out in a fan like pattern and shaped in sharp angled wedges.

Yeah, and his example about bugs in the kitchen. Everything is edible and 'wild' or 'heirloom' and "contarra ccetnxniams luryca tanniounons" comes from the farmers talking about ... vegetables. So there's a definite interrelationship between the 'words' and the images. I'm unconvinced by the rebuttal as well, not to say I am convinced we have a fully formal language going on here, but there's definitely some shared c…

His counterexamples also have a flaw. He's expecting that mixing two languages have a consistent result given the human language meaning. Those words might have meaning in the DALLE language that totally flips the meaning of the whole phrase. Each batch of images is internally consistent.

Re: DALL-E 2 has a secret language

#115

Post by ksaj »

astrange wrote: post #86
TOMDM wrote: post #41

Earlier quoted context omitted.

A uniform distribution makes sense for gibberish, not something I'd considered. A counterpoint I'd raise is I wonder how aggressive Dall-E 2 is in making assumptions about words it hasn't seen before. Hard to do given that it's read essentially the entire internet, however someone could make up some latin-esque words that people would be able to guess the meaning of. If the model is as good as people at assuming the…

Prior art with GPT2: https://www.thisworddoesnotexist.com/

This might be considerably different, and calling it "prior art" fails to consider what is actually going on here. The appearance may be similar, but lots of things can look similar while being completely distinct. And this is indeed such a case.

One of the words I got was "charlite" for the pale green colour of charcoal used as a dye. Charlite might not be a real word, but it is made up the same way a real word would be.

The method is important, because "charlite" probably came about by specifically asking GPT2 for a definition to the non-word "charlite."

In fact, this shows up in the source code examples:

# definition for a word you make up print(word_generator.generate_definition("glooberyblipboop"))

This is literally the opposite of what OP is presented, since we know where the "defined" word comes from with the GPT2 examples, which means that was a demo of GPT2 trying to work out a human provided word. It is literally a function of the program: generate_definition(). It was specifically written to do that.

But we don't know where the words come from, even though they are internally consistent, with the DALL-E 2 examples. As far as we can tell, it's an internal phenomenon not based on intentional human input.

Having said that, GPT2 probably has the same phenomenon. But the link you provided is not demonstrating that.

Re: DALL-E 2 has a secret language

#117

Post by burrows »

momojo wrote: post #78
TOMDM wrote: post #34

Shouldn't this be expected to a certain extent? Gibberish has to map _somewhere_ in the models concept space. Whether is maps onto anything we'd recognise as consistent doesn't mean that the AI wouldn't have some concept of where it relates, as other people have noted, the gibberish breaks down when you move it into another context, but who's to say that Dall-E 2 isn't remaining consistent to some concept it understa…

> Shouldn't this be expected to a certain extent? In hindsight, sure. Given enough time someone might have predicted the phenomenon. But I don't think most of us did. What's more fascinating to me is how often this has happened in this space in just the last few years. 1. Some phenomenon is discovered 2. I'm surprised 3. It makes sense in hindsight

NP

Re: DALL-E 2 has a secret language

#118

Post by jk7tarYZAQNpTQa »

civilized wrote: post #100
TOMDM wrote: post #34

Shouldn't this be expected to a certain extent? Gibberish has to map _somewhere_ in the models concept space. Whether is maps onto anything we'd recognise as consistent doesn't mean that the AI wouldn't have some concept of where it relates, as other people have noted, the gibberish breaks down when you move it into another context, but who's to say that Dall-E 2 isn't remaining consistent to some concept it understa…

> Gibberish has to map _somewhere_ in the models concept space. Why? It could just go to noise images, or vaguely real-looking objects that don't look like anything in particular.

Are these algorithms even capable of generating noise images? And I don't mean asking them to generate "an image of tv static".
Post reply on HN