I have a sneaky suspicion there is an if-then situation here - always the edge cases :).
No sneakiness. Fully open my friend. This is just a bit of fun.
GPT Unicorn has drawn a unicorn
171–180 of 207 posts
Re: GPT Unicorn has drawn a unicorn
#172Earlier quoted context omitted.
I'm not sure what you mean by random sampling. If I sample a random SVG, I wouldn't expect to look like anything, let alone roughly like a unicorn.
I mean random sampling in the sense how autoregressive language models like GPT generate sequences using token probabilities. It's not a random svg, but the text sequence that is used to draw it suffers from inherent non-determinism in the underlying model.
Relying on token probabilities seems like the exact opposite of random
Re: GPT Unicorn has drawn a unicorn
#173Earlier quoted context omitted.
I'm sorry it hurts your head. I'm happy to sponsor a packet of paracetamol or some water if that helps. Ultimately, this is fun, not science. I'm just happy that after all these attempts, it finally got to a unicorn.
Well done for being a good sport, but I'm willing to bet that tomorrow's shape will not resemble a unicorn and you'll have to figure out how that works with your assumption that the model is improving.
I think we're going to need to wait a few more years, at least, to see any improvement. I expect to see 4 new models a year, before GPT-5 arrives. I'll just keep using the latest model and we can all reconvene in 1, 2, 5, 10 years.
Re: GPT Unicorn has drawn a unicorn
#174Earlier quoted context omitted.
Then you could use mechanical turk to have them rate each image to figure out how close it is to a Unicorn...
We could but this is also a fun project, which is why when I checked it today I was surprised that what I saw was not a turd with eyes (2023-05-18) nor a strange sea creature (2023-07-08) but something which, for the first time I think, actually resembled a unicorn. I appreciate all the comments around determinism, sampling, scientific method, but as I said when I posted this just after building, it really is just fo…
You don’t have to justify yourself to the HN peanut gallery :-)
Re: GPT Unicorn has drawn a unicorn
#175What is strange to me about this is that with a remarkably similar prompt ("Draw a unicorn in SVG format") I'm able to get this from the Bing image creator (which is powered by DALL-E -- and I would think GPT-4 would have this capability. Perhaps I'm being naive): https://th.bing.com/th/id/OIG.GqpaRZ.NXCN6uxKj7X1u?pid=ImgGn
When you prompt GPT-4 with the API, you can only get text out. In this case OP is asking for text that happens to define an SVG file (which is a text format).
When you ask Bing Image Creator to produce an image, your prompts goes to the image model and an image comes out. It's a bitmap image, not an SVG—that unicorn is defined not as text but as a series of pixels with different colors.
Comparing the two is comparing apples and oranges, because there are completely different models underneath and completely different output formats.
Re: GPT Unicorn has drawn a unicorn
#176Earlier quoted context omitted.
> This comment section is a super fascinating case study on the inherent flaws in human cognition. Especially when it comes to seeing patterns in random noise. The fact that some people believe that the model really has to have changed in the past few days is amazing You need only to look at the discourse around the Tesla FSD superusers to see this: they report a glitch at an intersection one day, then believe the ne…
I noticed similar a behavior in Stable Diffusion forums where people believe that the model they downloaded and are running offline is getting better at understanding their prompts.
Stable diffusion most likely don't do it, but even static model that as an input takes embedding of all your historic prompts + current prompt would get progressively give you better inputs as you use it.
Re: GPT Unicorn has drawn a unicorn
#177Not directly related to the post, but still feels somewhat relevant.
Back in March I used a bit more elaborate multi-step prompts for GPT3.5 to generate amusing pictures and published a gallery [1]. However, I eventually reached a point where changing prompts did not consistently improve the final results. At the end of the day, the quality of images are only as good as the training dataset, and GPT is a black box.
For something different, to test whether it is possible to "compress" visual content specifically for GPT, I ran another experiment. SVG, being a verbose format, takes time to generate a detailed image, and it also becomes expensive over time. I translated a subset of SVG elements into Forth words [2], which has a nice synergy with GPT tokens--this allowed me to progressively render pictures and produce smaller outputs without sacrificing much in quality.
Finally, I training my own GPT2-like model on the QuickDraw dataset [3]. It's not surprising that a sequence transformer can be trained to produce coherent brush strokes and recognizable images as long as there is a way to translate a graphical content into a sequence of tokens. That said, I found myself with more questions than I started, and trying other ideas now.
[1] https://drawmeasheep.net/pages/about.html
Re: GPT Unicorn has drawn a unicorn
#178This is why, as a product manager, you should always test 20 hypotheses per month. At p-value of 0.05 this basically guarantees a successful product feature test every month!
Huh? Either I do not understand what you mean, or you do not understand probability (sorry). My understanding is that you wanted to test whether a certain feature improves user satisfaction or not. Assuming that users have a 0.05=1/20 probability of liking each feature, by testing 20 features you can get at least one successful feature (because 0.05x20=1). This is wrong in two ways. First, the p-value is the probabil…
Pretty sure it was a joke.
Re: GPT Unicorn has drawn a unicorn
#179Earlier quoted context omitted.
> This comment section is a super fascinating case study on the inherent flaws in human cognition. Especially when it comes to seeing patterns in random noise. The fact that some people believe that the model really has to have changed in the past few days is amazing You need only to look at the discourse around the Tesla FSD superusers to see this: they report a glitch at an intersection one day, then believe the ne…
Go into /r/chatgpt and /r/bing and it's a bit scary how many people anthropomorphize the models.
Interesting that modern discourse is to use "scary" and "dangerous" and such words so much more. I wonder if it is related to the present rise in neuroticism and trigger warnings etc.
It's not particularly "scary" to me that people do that. I remember Boomer and Scooby Doo bots that people anthropomorphized and those were warbots from barely 10 y ago.
I suppose, in today's parlance, "it's scary how much people use fear-oriented language for normal things".
Re: GPT Unicorn has drawn a unicorn
#180I was curious about how they were getting GPT to generate images, the prompting is so simple[1] that GPT still blows my mind:
async fetchImage(context) {
const messages = context || [
{ role: 'system', content: `You are a helpful assistant that generates SVG drawings. You respond only with SVG. You do not respond with text.` },
{ role: 'user', content: `Draw a unicorn in SVG format. Dimensions: 500x500. Respond ONLY with a single SVG string. Do not respond with conversation or codeblocks.` }
]
const response = await this.api.generateCompletion(messages)
if (!isSvg(response.content)) {
console.error('Generated image is not valid SVG:', response.content)
messages.push({ role: 'assistant', content: response.content })
messages.push({ role: 'user', content: `The generated image is not valid SVG. Please try again. Only respond with SVG code. No text.` })
return this.fetchImage(messages)
}
console.log('Generated image:', response.content)
return response
}