Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

adamkdean.co.uk

Post reply on HN
Print view View original

91–100 of 109 posts

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#91

Post by imdsm »

int_19h wrote: post #51
ratg13 wrote: post #25

Earlier quoted context omitted.

Aren’t they using the March 14 model like the general public? It’s frozen in time, there are no updates to it.. All of these will be drawn using the same model until they push a new update, or you switch to a different GPT But I already think they proved the point that the generation is random enough that it would be extremely difficult to track progress this way.

When I ask GPT-4 to draw a unicorn, I get this: https://i.imgur.com/HQ1Vjp0.png And when I ask it to draw Mona Lisa, I get this: https://i.imgur.com/gsBU49w.png So far as I can see, the only difference is that the GitHub repo uses the API; the ones that I have were rendered using the web chat UI. Which makes me wonder if they're using the bleeding edge model for the chat.

That's a really interesting question!

I haven't yet gained access to the enhanced chat features with image outputs. I'm using the API with default parameters, with the gpt-4-0314 model, outputting SVG.

It's not a perfect experiment, but we'll see how it gets on over time.

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#92

Post by imdsm »

atleastoptimal wrote: post #38
pps wrote: post #22

Earlier quoted context omitted.

"GPT 5 Will be Released 'Incrementally' - 5 Points from Brockman Statement" - https://www.youtube.com/watch?v=1NAmLp5i4Ps

gpt-4-0314 is a snapshot model and won't be updated, they shouldn't use that for this experiment.

The model being used is `gpt-4` but each result records the model that was used to generate the output. With incremental updates, the model will change automatically. This is to support incremental updates without needing anything from me. :)

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#93

Post by imdsm »

sacred_numbers wrote: post #23
dangond wrote: post #6

> The idea behind GPT Unicorn is quite simple: every day, GPT-4 will be asked to draw a unicorn in SVG format. This daily interaction with the model will allow us to observe changes in the model over time, as reflected in the output. Is it useful to do this every day? Correct me if I'm wrong, but my understanding is that OpenAI does not update the models available in production incrementally on a day-to-day basis.

They do update the model in the background, although I'm not sure how often or how much they update it. To avoid issues with this practice they offer gpt-4-0314 which says this in the documentation: "Snapshot of gpt-4 from March 14th 2023. Unlike gpt-4, this model will not receive updates, and will only be supported for a three month period ending on June 14th 2023." Unfortunately this experiment is using the frozen…

Author here: these images are using `gpt-4` but I'm recording the specific model that OpenAI use with each result. As the incremental updates come out, that will change (without requiring me to change anything.)

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#94

Post by imdsm »

qumpis wrote: post #35
m3kw9 wrote: post #34

Are they banking on OpenAI updating their model every day, or just prompting the same thing everyday wishing for a different outcome?

In the "sparks of AGI" paper, authors noted that the unicorn shape degrees as more "alignment" is injected to to. If openai adjust the model (say by training more), the picture should reflect it. If they make the model be more "aligned", it should reflect as well. So I'd guess the answer is the former.

The former yes. The expectation is that updates will start to become more and more iterative over time. For this reason, I record the specific version that is pegged to the generic `gpt-4` model.

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#95

Post by imdsm »

anon291 wrote: post #73

How can a model trained on text know what a unicorn looks like

That is the beauty of the model, that it can understand what a unicorn looks like, and then know roughly where to place the features. Currently we're seeing poor results due to safety tuning but in time, it'll be very interesting to see how it improves.

There are a few factors at play here: knowing what a unicorn looks like, knowing the different areas of a unicorn, being able to translate that into a 2D space, and being able to form the connection between code (language) and appearance.

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#96

Post by imdsm »

renewiltord wrote: post #66

Hmm, tempting to make a thing that just loops: description = gpt4("describe ") and chain that into new_result = gpt4(" was described by a human as but it should have been visibly apparent as a unicorn. provide SVG output that ") and post that as today's output. I wonder what we'll get. I'll do it manually.

Now that is an interesting thought!

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#97

Post by simonw »

imdsm wrote: post #90
easygenes wrote: post #88

Earlier quoted context omitted.

Greg Brockman just stated in a long post on Twitter yesterday: "...it’s easy to create a continuum of incrementally-better AIs (such as by deploying subsequent checkpoints of a given training run), which presents a safety opportunity very unlike our historical approach of infrequent major model upgrades." [1] This implies OpenAI will be shifting strategy to incrementally releasing future models, so we won't just sudd…

Author here — thanks for explaining this. My expectation is that there will be incremental updates to the model, so while I'm providing the model `gpt-4` for completions, I'm recording the actual model, `gpt-4-0314` in this case, along with the result. I don't want to monitor (and potentially miss) model updates, which is likely as this is very much a fire and forget project that I'll review over time. One per day se…

But if each image is random, what does an image being "better" one day actually tell you about the underlying model?

That's why I suggested generating 16 models each time instead of just 1 a- if all 16 are noticeably better than the previous day you've learned something a lot more interesting than if just one appears to be better than the previous one.

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#98

Post by imdsm »

charcircuit wrote: post #12
dangond wrote: post #6

> The idea behind GPT Unicorn is quite simple: every day, GPT-4 will be asked to draw a unicorn in SVG format. This daily interaction with the model will allow us to observe changes in the model over time, as reflected in the output. Is it useful to do this every day? Correct me if I'm wrong, but my understanding is that OpenAI does not update the models available in production incrementally on a day-to-day basis.

Similarly the quality of the model can't be judged with a single sample. These end up canceling out.

I updated my blog post with clarifications but I expect to see two types of changes over time, the variance of output, and the quality of output.

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#99

Post by imdsm »

Mystery-Machine wrote: post #24

Would be great if these days would have dates as well. Otherwise, there's little use of "Day 69". If I could see "Day 69 (June 21 2023)"

Currently it shows Today (which is always today) and then the image name, `image-2023-04-13` etc. I didn't think to make it any prettier than that, Day X etc. I could do!

Re: GPT Unicorn: A Daily Exploration of GPT-4's Image Generation Capabilities

#100

Post by imdsm »

startupsfail wrote: post #8

Sampling a single noisy sample from a model that doesn’t update that often is hardly correlated with the claim of “Daily exploration”.

Perhaps. I think right now the sample of 5 days isn't anything we can draw conclusions from, but given 365 days, with a number of model updates in between, we may see more interesting patterns.
Post reply on HN