Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

4o Image Generation

openai.com

Post reply on HN
Print view View original

441–450 of 629 posts

Re: 4o Image Generation

#441

Post by zaptrem »

fpgaminer wrote: post #296
xg15 wrote: post #162

Earlier quoted context omitted.

> What's important about this new type of image generation that's happening with tokens rather than with diffusion That sounds really interesting. Are there any write-ups how exactly this works?

There are a few different approaches. Meta documents at least one approach quite well in one of their llama papers. The general gist is that you have some kind of adapter layers/model that can take an image and encode it into tokens. You then train the model on a dataset that has interleaved text and images. Could be webpages, where images occur in-between blocks of text, chat logs where people send text messages and…

Token by token diffusion was done by MAR https://arxiv.org/abs/2406.11838 and Fluid (scaled up MAR) https://arxiv.org/abs/2410.13863

When you randomly pick the locations they found it worked okay, but doing it in raster order (left to right, top to bottom) they found it didn't work as well. We tried it for music and found it was vulnerable to compounding error and lots of oddness relating to the fragility of continuous space CFG.

Re: 4o Image Generation

#442

Post by mgraczyk »

noosphr wrote: post #438
mgraczyk wrote: post #429

Earlier quoted context omitted.

This is overly cynical. Models typically do know what tools they have access to because the tool descriptions are in the prompt. Asking a model which tools it has is a perfectly reasonable way of learning what is effectively the content of the prompt. Of course the model may hallucinate, but in this case it takes a few clicks in the dev tools to verify that this is not the case.

>Of course the model may hallucinate, but in this case it takes a few clicks in the dev tools to verify that this is not the case. I don't know - or care to figure out - how OpenAI does their tool calling in this specific case. But moving tool calls to the end user is _monumentally_ stupid for the latency if nothing else. If you centralize your function calls to a single model next to a fat pipe it means that you hal…

It's not client side, the messages are in the api though.

But what do you mean you don't care? The thing you were responding to was literally a claim that it was a tool call rather than direct output

Re: 4o Image Generation

#443

Post by mgraczyk »

noosphr wrote: post #433
mgraczyk wrote: post #427

Earlier quoted context omitted.

The tool name is not relevant. It isn't the actual name, they use an obfuscated name. The fact that the model believes it is a tool is good evidence at first glance that it is a tool, because the tool calls are typically IN THE PROMPT. You can literally look at the JavaScript on the web page to see this. You've overcorrected so far in the wrong direction that you think anything the model says must be false, rather th…

>The tool name is not relevant. It isn't the actual name, they use an obfuscated name. >EDIT: And googling the tool name I see it's already been widely discussed on twitter and elsewhere I am so confused by this thread.

The original claim was that the new image generation is direct multimodal output, rather than a second model. People provided evidence from the product, including outputs of the model that indicate it is likely using a tool. It's very easy to confirm that that's the case in the API, and it's now widely discussed elsewhere.

It's possible the tool is itself just gpt4o, wrapped for reliability or safety or some other reason, but it's definitely calling out at the model-output level

Re: 4o Image Generation

#444

Post by raincole »

jay_kyburz wrote: post #416
nestorD wrote: post #382

Earlier quoted context omitted.

Short answer: the model is good at consistency. You can use it to generate a set a style reference images, then use those as reference for all your subsequent generations. Generating in the same chat might also help it have further consistency between images.

Sorry that wasn't a question, I was saying they models were not good at constancy in my evaluation.

In terms of prompt adherence and consistency, the current state of art just changed dramatically today and you're in the very thread about the change.

Your evaluation, done a few weeks ago, isn't relevant anymore.

Re: 4o Image Generation

#445

Post by throwaway314155 »

Taek wrote: post #242
minimaxir wrote: post #2

OpenAI's livestream of GPT-4o Image Generation shows that it is slowwwwwwwwww (maybe 30 seconds per image, which Sam Altman had to spin "it's slow but the generated images are worth it"). Instead of using a diffusion approach, it appears to be generating the image tokens and decoding them akin to the original DALL-E ( https://openai.com/index/dall-e/ ), which allows for streaming partial generations from top to botto…

> it appears to be generating the image tokens and decoding them akin to the original DALL-E The animation is a lie. The new 4o with "native" image generating capabilities is a multi-modal model that is connected to a diffusion model. It's not generating images one token at a time, it's calling out to a multi-stage diffusion model that has upscalers. You can ask 4o about this yourself, it seems to have a strong under…

Sorry but I think you may be mistaken if your only source is ChatGPT. It's not aware of its own creation processes beyond what is included in its system prompt.

Re: 4o Image Generation

#446

Post by petesergeant »

oortoo wrote: post #295
nmilo wrote: post #288

Visual internet content is completely over. Pack it up

For starters, this completely blocks generation of anything remotely related to copy-protected IPs, which may actually be a saving grace for some creatives. There's a lot of demand for fanart of existing characters, so until this type of model can be run locally, the legal blocks in place actually give artists some space to play in where they don't have to compete with this. At least for a short while.

I’ve had it do Tintin and the Simpsons in the last hour, so no, it doesn’t

Re: 4o Image Generation

#448

Post by jay_kyburz »

raincole wrote: post #444
jay_kyburz wrote: post #416

Earlier quoted context omitted.

Sorry that wasn't a question, I was saying they models were not good at constancy in my evaluation.

In terms of prompt adherence and consistency, the current state of art just changed dramatically today and you're in the very thread about the change. Your evaluation, done a few weeks ago, isn't relevant anymore.

I don't see any evidence of that, and in fact, in the video shows the style moving all over the place.

I look forward to giving it a try, but I don't have high hopes.

Re: 4o Image Generation

#449

Post by RobinL »

It's very impressive. It feels like the text is a bit of a hack where they're somehow rendering the text separately and interpolating it into the image. Not always, I got it to render calligraphy with flourishes, but only for a handful of words.

For example, I asked it to render a few lines of text on a medieval scroll, and it basically looked like a picture of a gothic font written onto a background image of a scroll

Re: 4o Image Generation

#450

Post by aurareturn »

First AI image generator to pass the uncanny valley test? Seems like it. This is the biggest leap in image generation quality I've ever seen.

How much longer until an AI that can generate 30 frames with this quality and make a movie?

About 1.5 years ago, I thought AI would eventually allow anyone with an idea to make a Hollywood quality movie. Seems like we're not too far off. Maybe 2-3 more years?

Post reply on HN