Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

adamkdean.co.uk

Post reply on HN
Print view View original

101–109 of 109 posts

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#101

Post by imdsm »

simonw wrote: post #97
imdsm wrote: post #90

Earlier quoted context omitted.

Author here — thanks for explaining this. My expectation is that there will be incremental updates to the model, so while I'm providing the model `gpt-4` for completions, I'm recording the actual model, `gpt-4-0314` in this case, along with the result. I don't want to monitor (and potentially miss) model updates, which is likely as this is very much a fire and forget project that I'll review over time. One per day se…

But if each image is random, what does an image being "better" one day actually tell you about the underlying model? That's why I suggested generating 16 models each time instead of just 1 a- if all 16 are noticeably better than the previous day you've learned something a lot more interesting than if just one appears to be better than the previous one.

I don't think any individual sample will tell us much. I think we'll need to review this in 3 months, 6 months, 12 months etc, and look for patterns of changes. Ultimately, this is just a fun side project, something to leave running and check back on once every month or two.

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#104

Post by easygenes »

simonw wrote: post #97
imdsm wrote: post #90

Earlier quoted context omitted.

Author here — thanks for explaining this. My expectation is that there will be incremental updates to the model, so while I'm providing the model `gpt-4` for completions, I'm recording the actual model, `gpt-4-0314` in this case, along with the result. I don't want to monitor (and potentially miss) model updates, which is likely as this is very much a fire and forget project that I'll review over time. One per day se…

But if each image is random, what does an image being "better" one day actually tell you about the underlying model? That's why I suggested generating 16 models each time instead of just 1 a- if all 16 are noticeably better than the previous day you've learned something a lot more interesting than if just one appears to be better than the previous one.

I suggest the author generate a few different examples per day, each with a different temperature setting. 0.0 temperature ought to be deterministic on the same model, but would be interesting to look for trends at higher temperatures over time.

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#105

Post by imdsm »

easygenes wrote: post #104
simonw wrote: post #97

Earlier quoted context omitted.

But if each image is random, what does an image being "better" one day actually tell you about the underlying model? That's why I suggested generating 16 models each time instead of just 1 a- if all 16 are noticeably better than the previous day you've learned something a lot more interesting than if just one appears to be better than the previous one.

I suggest the author generate a few different examples per day, each with a different temperature setting. 0.0 temperature ought to be deterministic on the same model, but would be interesting to look for trends at higher temperatures over time.

That's a great idea, thanks easygenes!

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#106

Post by whistle650 »

simonw wrote: post #55

I filed an issue: https://github.com/adamkdean/gpt-unicorn/issues/2 "Running this project daily doesn't make sense if GPT-4 is not being constantly updated" With a suggestion to run it monthly instead, and generate 16 images at a time, and backfill it for GPT3 and GPT3.5.

In the talk he specifically mentions the very interesting fact that as they improved "alignment" it affected the unicorn output (negatively if I remember correctly). So as long as "alignment" is changing, the output should change. Not sure but ongoing RLHF, changing "system" prompts etc can and do change while the underlying foundational model need not.

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#107

Post by anon291 »

imdsm wrote: post #95
anon291 wrote: post #73

How can a model trained on text know what a unicorn looks like

That is the beauty of the model, that it can understand what a unicorn looks like, and then know roughly where to place the features. Currently we're seeing poor results due to safety tuning but in time, it'll be very interesting to see how it improves. There are a few factors at play here: knowing what a unicorn looks like, knowing the different areas of a unicorn, being able to translate that into a 2D space, and b…

How though? I have tried to get got to generate vector art and my attempts usually failed with the model getting stuck into recurring patterns with no structure. I'm surprised anyone was able to get it to generate something.

In particular, I cannot understand how the models can properly understand concepts such as spatial relations without being able to 'see'

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#108

Post by beanbean01 »

sdenton4 wrote: post #67
flangola7 wrote: post #58

Earlier quoted context omitted.

I don't see how that diagnoses them as unqualified. The conclusion is unsupported.

Since 'gets anything wrong, ever' is the current goalpost for agi (per the Gary Marcus methodology), we have to judge human intelligence by the same stick. Since the author of this article misunderstood the gpt release process, they have proven they are a non sentient pile of trash brain, ready to be processed into hamburger.

Actually ... that's a reasonable goalpost, in my opinion. Yes, humans make careless mistakes. However, humans mostly make careless mistakes because A) their brains have to reconstruct information every time they use it, or B) they are tired. LLMs, as piles of linear algebra, have neither excuse. Their training data is literally baked into their weights, and linear algebra does not get tired.

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#109

Post by int_19h »

imdsm wrote: post #91
int_19h wrote: post #51

Earlier quoted context omitted.

When I ask GPT-4 to draw a unicorn, I get this: https://i.imgur.com/HQ1Vjp0.png And when I ask it to draw Mona Lisa, I get this: https://i.imgur.com/gsBU49w.png So far as I can see, the only difference is that the GitHub repo uses the API; the ones that I have were rendered using the web chat UI. Which makes me wonder if they're using the bleeding edge model for the chat.

That's a really interesting question! I haven't yet gained access to the enhanced chat features with image outputs. I'm using the API with default parameters, with the gpt-4-0314 model, outputting SVG. It's not a perfect experiment, but we'll see how it gets on over time.

To be clear, I'm not using multimodal chat, either. I just told it to output an SVG snippet in the chat and then manually copy/pasted it to view.
Post reply on HN