Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No elephants: Breakthroughs in image generation

oneusefulthing.org

Post reply on HN
Print view View original

21–30 of 373 posts

Re: No elephants: Breakthroughs in image generation

#21

Post by [deleted] »

OtherShrezzing wrote: post #19
shkkmo wrote: post #15

Earlier quoted context omitted.

> This was an important discussion to have two to three years ago, then we had it online, and then we more or less agreed that it's unfair for artists to have their works sucked up with no recourse. Speak for yourself, there was no consensus online. There are plenty of us that think that dramatically expanding the power of copyright would be a huge mistake that would primarily benefit larger companies and do little t…

>There are plenty of us that think that dramatically expanding the power of copyright would be a huge mistake that would primarily benefit larger companies and do little to protect or fund small artists. The status quo also primarily benefits larger companies, and does little (exactly nothing, if we're being earnest) to protect or fund small artists. It's reasonable to hold both opinions that: 1) artists aren't being…

[deleted]

Re: No elephants: Breakthroughs in image generation

#22

Post by WhiteNoiz3 »

Retr0id wrote: post #4

I had a reasonable intuition for how the "old" method works, but I still don't grok this new approach. "in multimodal image generation, images are created in the same way that LLMs create text, a token at a time" Is there some way to visualise these "image tokens", in the same way I can view tokenized text?

I haven't see any details on how OpenAI's model works, but the tokens it generates aren't directly translated into pixels - those tokens are probably fed into a diffusion process which generates the actual image.. The tokens are the latent space or conditioning for the actual image generation process.

Re: No elephants: Breakthroughs in image generation

#23

Post by eadmund »

probably_wrong wrote: post #12

> Is it okay to reproduce the hard-won style of other artists using AI? Who owns the resulting art? Who profits from it? Which artists are in the training data for AI, and what is the legal and ethical status of using copyrighted work for training? These were important questions before multimodal AI, but now developing answers to them is increasingly urgent. I have to disagree with the conclusion. This was an importa…

> it's unfair for artists to have their works sucked up

I never thought it was unfair to artists for others to look at their work and imitate it. That seems to me to be what artists have been doing since the second caveman looked at a hand painting on a cave wall and thought, ‘huh, that’s pretty neat! I’d like to try my hand at that!’

Re: No elephants: Breakthroughs in image generation

#24

Post by orbital-decay »

4o still exhibits the "pink elephant effect", it's just... subtler, and tends to reveal itself on a complex or confusing prompt. Negations are also still not handled properly, they tend to slightly confuse the model and decrease the accuracy of the answer or the generated picture. The same is true for any other LLM. Moreover, the author is asking the model to rationalize the decision he already made ("tell me why there can't be any elephants"), which could work as an equivalent to a CoT step.

It's "just" a much bigger and much better trained model. Which is a quality on its own, absolutely no doubt about that. Fundamentally the issue is still there though, just less prominent. Which kind of makes sense - imagine the prompt "not green", what even is that? It's likely slightly out of distribution and requires representing a more complex abstraction, so the accuracy will necessarily be worse than stating the range of colors directly. The result might be accurate, until the model is confused/misdirected by something else, and suddenly it's not.

I think in the end none of the architectural differences will matter beyond the scaling. What will matter a lot more is data diversity and training quality.

Re: No elephants: Breakthroughs in image generation

#25

Post by rob74 »

bla3 wrote: post #13
card_zero wrote: post #7

Looking at the example where the coffee table is swapped, I notice every time the image is reprocessed it mutates, based on the previous iteration, and objects become more bizarre each time, like chinese whispers. * The weird-ass basket decoration on the table originally has some big chain links (maybe anchor chain, to keep the theme with the beach painting). By the third version, they're leathery and are merging wit…

The pictures on the wall change too.

Actually, almost everything changes slightly - the number, shape and pattern of the chairs, the number and pattern of the pillows, the pattern of the curtains, the scene outside the window, the wooden part of the table, the pattern of the carpet... The blue couch stays largely the same, it just loses some detail...

Re: No elephants: Breakthroughs in image generation

#26

Post by M4v3R »

card_zero wrote: post #7

Looking at the example where the coffee table is swapped, I notice every time the image is reprocessed it mutates, based on the previous iteration, and objects become more bizarre each time, like chinese whispers. * The weird-ass basket decoration on the table originally has some big chain links (maybe anchor chain, to keep the theme with the beach painting). By the third version, they're leathery and are merging wit…

Yeah, this is in my opinion the biggest limitation of the current gen GPT 4o image generation: it is incapable of editing only parts of an image. I assume what it does every time is tokenizing the source image, then transforming it according to the prompt and then giving you the final result. For some use cases that’s fine but if you really just want a small edit while keeping the rest of the image intact you’re out of luck.

Re: No elephants: Breakthroughs in image generation

#27

Post by d4rkp4ttern »

Diagrams are still a big unsolved problem. Making diagrams for a talk or paper is an extremely tedious process and I am still waiting for a good multimodal LLM solution for this. It should take a sketch and/or text description of what you want and in a few iterations you should get what you want. GPT4o tries hard but results are still bad.

Re: No elephants: Breakthroughs in image generation

#28

Post by shubhamjain »

The Ghibli trend completely missed the real breakthrough — and it’s this. The ability to closely follow text, understand the input image, and maintain context of what’s already there is a massive leap in image generation. While Midjourney delivered visually stunning results, I constantly struggled to get anything specific out of it, making it pretty much useless for actual workflows.

4o is the first image generation model that feels genuinely useful not just for pretty things. It can produce comics, app designs, UI mockups, storyboards, marketing assets, and so on. I saw someone make a multi-panel comic with it with consistent characters. Obviously, it's not perfect. But just getting there 90% is a game changer.

Re: No elephants: Breakthroughs in image generation

#30

Post by BriggyDwiggs42 »

eadmund wrote: post #23
probably_wrong wrote: post #12

> Is it okay to reproduce the hard-won style of other artists using AI? Who owns the resulting art? Who profits from it? Which artists are in the training data for AI, and what is the legal and ethical status of using copyrighted work for training? These were important questions before multimodal AI, but now developing answers to them is increasingly urgent. I have to disagree with the conclusion. This was an importa…

> it's unfair for artists to have their works sucked up I never thought it was unfair to artists for others to look at their work and imitate it. That seems to me to be what artists have been doing since the second caveman looked at a hand painting on a cave wall and thought, ‘huh, that’s pretty neat! I’d like to try my hand at that!’

Right the difference is that it’s a large company looking at it then copying it and reselling it without credit, which basically everyone would understand as bad without the indirection of a model.

Edit: the key words here are “company” and “reselling”

Post reply on HN