101–110 of 118 posts
[1] https://twitter.com/barneyflames/status/1531736708903051265?...
Earlier quoted context omitted.
Is it gibberish in the true sense, or is it some sort of AI generated/learned latin text for the input models used? I wonder if they used a large number of biological images in their training data along with their scientific names, which led to this second order effect.
I think calling it gibberish is a misnomer, it would be gibberish if inconsistent, but if the same strings of characters lead to the same semantic objects then that is not gibberish.
Spoilers:
These AI controlled soldiers have developed their own language (communicated wirelessly), and the humans think that they are just mute.
This one melts my brain a bit, I’m not going to lie. Whales talking about food, with subtitles. “Translate” the subtitles and you get food that whales would actually eat.
According to [1], the byte pair encoding for “Apoploe vesrreaitais” (the words producing bird images) is "apo, plo, e , ,ve, sr, re, ait, ais ", and Apo-didae & Plo-ceidae are families of birds. [1] https://twitter.com/barneyflames/status/1531736708903051265?...
According to [1], the byte pair encoding for “Apoploe vesrreaitais” (the words producing bird images) is "apo, plo, e , ,ve, sr, re, ait, ais ", and Apo-didae & Plo-ceidae are families of birds. [1] https://twitter.com/barneyflames/status/1531736708903051265?...
According to [1], the byte pair encoding for “Apoploe vesrreaitais” (the words producing bird images) is "apo, plo, e , ,ve, sr, re, ait, ais ", and Apo-didae & Plo-ceidae are families of birds. [1] https://twitter.com/barneyflames/status/1531736708903051265?...
On the other hand the openai tokenizer gives me a different tokenization ap - opl - oe [0]. If you capitalize A the result is A - pop - loe. The dalle 2 paper only specifies that it uses a BPE encoding, I would assume they used the same one as for gpt3 [0] https://beta.openai.com/tokenizer
And for the record, they use BPE dropout for DALLE-1, see https://arxiv.org/pdf/2102.12092.pdf
Earlier quoted context omitted.
On the other hand the openai tokenizer gives me a different tokenization ap - opl - oe [0]. If you capitalize A the result is A - pop - loe. The dalle 2 paper only specifies that it uses a BPE encoding, I would assume they used the same one as for gpt3 [0] https://beta.openai.com/tokenizer
If they use BPE dropout, then the split can be different and not unique. And for the record, they use BPE dropout for DALLE-1, see https://arxiv.org/pdf/2102.12092.pdf
Earlier quoted context omitted.
If they use BPE dropout, then the split can be different and not unique. And for the record, they use BPE dropout for DALLE-1, see https://arxiv.org/pdf/2102.12092.pdf
I believe they only apply it during training.
Shouldn't this be expected to a certain extent? Gibberish has to map _somewhere_ in the models concept space. Whether is maps onto anything we'd recognise as consistent doesn't mean that the AI wouldn't have some concept of where it relates, as other people have noted, the gibberish breaks down when you move it into another context, but who's to say that Dall-E 2 isn't remaining consistent to some concept it understa…
The machine is always trying to associate the words with other words semantically close together. E.g. when taken as input strong_man, or strng_man or srong_man these are all mean the same because that combination of letters are usually used with the word man, and there is no other competitor word to replace the srong except strong.
Now why that should be considered a secret language, it is beyond me. The input language for the machine is a natural human language, and that means it is very poor defined language for the machine to recognize. That is going always to produce a lot of gibberish.