181–190 of 201 posts
Earlier quoted context omitted.
> It is a machine learning algorithm. True > It is an electronic Parrot. This is incorrect, and unclear why people think this. The whole point of a good ML system is that it doesn't parrot training data. A good system can extrapolate novel answers from things it has seen. That is very far from "parroting".
Don't listen to these bots, they're all just parroting this same idea that models parrot.
Earlier quoted context omitted.
> fundamental to how it works not a flaw that can be iterated away. Can you elaborate?
Dalle is trying to optimise a bunch of random pixels to meet the prompt. It has a strong sense of what 1 object is. Ask it for a photo of a kitten and you get an jaw droppingly realistic photo of a kitten. Ask it for a picture of 6 kittens and you get, well, whatever this is: https://labs.openai.com/s/PIZJe6GCfat9soN3WOV3eC9p 7 kittens, none well defined. Because the loss function it is optimising taps out once it ha…
Recall that it's stored a compressed version of TBs of images.
Earlier quoted context omitted.
> fundamental to how it works not a flaw that can be iterated away. Can you elaborate?
Dalle is trying to optimise a bunch of random pixels to meet the prompt. It has a strong sense of what 1 object is. Ask it for a photo of a kitten and you get an jaw droppingly realistic photo of a kitten. Ask it for a picture of 6 kittens and you get, well, whatever this is: https://labs.openai.com/s/PIZJe6GCfat9soN3WOV3eC9p 7 kittens, none well defined. Because the loss function it is optimising taps out once it ha…
Now, your examples simply show some slight artifacts and lack of details on specific things. You're presenting remaining shortcomings on these metrics as "fundamental to how it works not a flaw that can be iterated away", when in fact they have mostly been iterated away over the past few years.
Earlier quoted context omitted.
Dalle is trying to optimise a bunch of random pixels to meet the prompt. It has a strong sense of what 1 object is. Ask it for a photo of a kitten and you get an jaw droppingly realistic photo of a kitten. Ask it for a picture of 6 kittens and you get, well, whatever this is: https://labs.openai.com/s/PIZJe6GCfat9soN3WOV3eC9p 7 kittens, none well defined. Because the loss function it is optimising taps out once it ha…
I'm not an expert on AI, but your complaints sound like minor versions of the major problems that these image generation AI's had a couple of years ago. It used to be that they could only create a mishmash of textures reminiscent of the subject and style, and struggled creating distinct objects at all. Now, your examples simply show some slight artifacts and lack of details on specific things. You're presenting remai…
Earlier quoted context omitted.
Wait what? Glue as in extract high level semantic representations from _syntatic probabilities_ and pass on to appropriate domain specific tools? This is the glaring hole in LLMs, a paradoxical semantic incoherence despite impressive sentenial and gramatical coherence. As glue it is so thin as to be potable.
you would still need an executive it would be more like a universal translator.
Earlier quoted context omitted.
Dalle is trying to optimise a bunch of random pixels to meet the prompt. It has a strong sense of what 1 object is. Ask it for a photo of a kitten and you get an jaw droppingly realistic photo of a kitten. Ask it for a picture of 6 kittens and you get, well, whatever this is: https://labs.openai.com/s/PIZJe6GCfat9soN3WOV3eC9p 7 kittens, none well defined. Because the loss function it is optimising taps out once it ha…
If you ranked its training data by similarity to your "witchcraft" images, it wouldnt seem witchraft. Recall that it's stored a compressed version of TBs of images.
Earlier quoted context omitted.
There are quite simple tricks to avoid repetition/copying in NNs, e.g. by (1) training a model to predict the "popularity" of the main model's outputs and penalizing popular/copied productions by backpropping through that model so as to decrease the predicted popularity, or (2) by conditioning on random inputs (LLMs can be prompted with imaginary "ID XXX" prefixes before each example to mitigate repetitions), or (3)…
I'm not sure what you mean by "backpropping through that model so as to decrease the predicted popularity". During training, we train a model to literally reproduce famous chunks of music exactly as they are in the training set. We can also learn to predict popularity at the same time, but we can't backpropagate anything that will reduce popularity, because this would directly contradict the main loss objective of ex…
Earlier quoted context omitted.
I'm not sure what you mean by "backpropping through that model so as to decrease the predicted popularity". During training, we train a model to literally reproduce famous chunks of music exactly as they are in the training set. We can also learn to predict popularity at the same time, but we can't backpropagate anything that will reduce popularity, because this would directly contradict the main loss objective of ex…
What I had in mind was kind of like a reward model that is trained by on longer outputs that have a very high similarity to training examples. Something similar has been done to prevent LLMs from using toxic language. You'd simply backprop through that model like in GANs. And no it does not contradict the overall training objective completely because the criterion would be long verbatim copies and it would not affect…
Earlier quoted context omitted.
Dalle is trying to optimise a bunch of random pixels to meet the prompt. It has a strong sense of what 1 object is. Ask it for a photo of a kitten and you get an jaw droppingly realistic photo of a kitten. Ask it for a picture of 6 kittens and you get, well, whatever this is: https://labs.openai.com/s/PIZJe6GCfat9soN3WOV3eC9p 7 kittens, none well defined. Because the loss function it is optimising taps out once it ha…
I'm not an expert on AI, but your complaints sound like minor versions of the major problems that these image generation AI's had a couple of years ago. It used to be that they could only create a mishmash of textures reminiscent of the subject and style, and struggled creating distinct objects at all. Now, your examples simply show some slight artifacts and lack of details on specific things. You're presenting remai…
A larger corpus allows it to be more varied but it doesn't fix the fundamental limitations.