Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Imagen, a text-to-image diffusion model

gweb-research-imagen.appspot.com

Post reply on HN
Print view View original

611–620 of 661 posts

Re: Imagen, a text-to-image diffusion model

#611

Post by visarga »

gwern wrote: post #394
visarga wrote: post #12

Interesting discovery they made > We show that scaling the pretrained text encoder size is more important than scaling the diffusion model size. There seems to be an unexpected level of synergy between text and vision models. Can't wait to see what video and audio modalities will add to the mix.

I think that's unsurprising. With DALL-E 1, for example, scaling the VAE (the image model generating the actual pixels) hits very fast diminishing returns, and all your compute goes into the 'text encoder' generating the token sequence. Particularly as you approach the point where the image quality itself is superb and people increasingly turn to attacking the semantics & control of the prompt to degrade the quality…

I was thinking more about our traditional ResNet50 trained on ImageNet vs CLIP. ResNet was limited to a thousand classes and brittle. CLIP can generalise to new concept combinations with ease. That changes the game, and the jump is based on NLP.

Re: Imagen, a text-to-image diffusion model

#612

Post by visarga »

espadrine wrote: post #574
ravi-delia wrote: post #441

Earlier quoted context omitted.

Basically makes sense, no? DALLE-2 suffered from misunderstanding propositional logic, treating prompts as less structured then it should have. That's a text model issue! Compared to that, scaling up the image isn't as important (especially with a few passes).

Is there a way to confirm that this extra processing relates to the language structure, and not the processing of concepts? I wouldn’t be surprised if the lack of video and 3D understanding in the image dataset training fails to understand things like the fear of heights, and the concept of gravity ends up being learned in the text processing weights.

I am sure the image-text-video-audio-games model will come soon. The recent Gato was one step in that direction. There's so much video content out there, it begs for modelling. I think robotics applications will benefit the most from video.

Re: Imagen, a text-to-image diffusion model

#613

Post by howinteresting »

tomp wrote: post #531
howinteresting wrote: post #418

Earlier quoted context omitted.

Well that's not the issue here, the problem is the examples like searches for images of "unprofessional hair" returning mostly Black people in the results. That is something we can judge as objectively morally bad.

Did you see the image in the linked article? Clearly the “unprofessional hair” are people with curly hair. Some are white! It’s not the algorithm’s fault that P(curly|black) > P(curly|white).

It absolutely is the responsibility of the people making the algorithm available to the general public.

Re: Imagen, a text-to-image diffusion model

#614

Post by ccbccccbbcccbb »

SemanticStrengh wrote: post #604
ccbccccbbcccbb wrote: post #247

Earlier quoted context omitted.

Yep, the meat-space is generally a bit less woke than HN, so thanks for the reminder ))

Smoking these meats! https://youtu.be/YeemJlrNx2Q

Smoking them meats with his wifi! That explains some obvious anomalies in the meat-space pretty neatly)

Re: Imagen, a text-to-image diffusion model

#615

Post by jillesvangurp »

dirkc wrote: post #585
jillesvangurp wrote: post #538

Earlier quoted context omitted.

He was definitely onto something in that book where people also resort to using blockchains to fingerprint their behavior and build an unbreakable chain of authenticity. Later in that book that is used to authorize the hardware access of the deceased and uploaded individuals. A bit far out there in terms of plot but the notion of authenticating based on a multitude of factors and fingerprints is not that strange. We'…

> blockchains to fingerprint their behavior and build an unbreakable chain of authenticity. Later in that book that is used to authorize the hardware access of the deceased and uploaded individuals. maybe I misunderstood, but I had it that people used generative AI models that would transform the media they produced. The generated content can be uniquely identified, but the creator (or creators) retains anonymity. La…

All part of the mix. But definitely some blockchain thing underneath to tie it all together. Stephenson was writing about crypto currencies as early as the nineties. Around the time he also coined the term Metaverse.

Re: Imagen, a text-to-image diffusion model

#616

Post by Siira »

interblag wrote: post #587
discmonkey wrote: post #421

For people complaining that they can't play with the model... I work at Google and I also can't play with the model :'(

I think they address some of the reasoning behind this pretty clearly in the write-up as well? > The potential risks of misuse raise concerns regarding responsible open-sourcing of code and demos. At this time we have decided not to release code or a public demo. In future work we will explore a framework for responsible externalization that balances the value of external auditing with the risks of unrestricted open-…

> harmful stereotypes and representations

So we can't have this model because of ... the mere possibility of stereotypes? With this logic, humans should all die, as we certainly encode some nasty stereotypes in our brains.

This level of dishonesty to not give back to the community is not unexpected at this point, but seeing apologists here is.

Re: Imagen, a text-to-image diffusion model

#617

Post by w1nk »

xtreme wrote: post #598
w1nk wrote: post #581

Earlier quoted context omitted.

To expand a bit for the grandparent, if you check out this authors other repos you'll notice they have a thing for implementing these papers (multiple DALLE-2 implementations for instance). You should expect to see an implementation there pretty quickly I'd guess.

Not to diminish their contribution but implementing the model is only one third of the battle. The rest is building the training dataset and training the model on a big computer.

You're not wrong that the dataset and compute are important, and if you browse the author's previous work, you'll see there are datasets available. The reproduction of DALL-E 2 required a dataset of similar size to the one imagen was trained on (see: https://arxiv.org/abs/2111.02114).

The harder part here will be getting access to the compute required, but again, the folks involved in this project have access to lots of resources (they've already trained models of this size). We'll likely see some trained checkpoints as soon as they're done converging.

Re: Imagen, a text-to-image diffusion model

#618

Post by gpt5 »

Tehdasi wrote: post #353
ALittleLight wrote: post #284

Interesting to me that this one can draw legible text. DALLE models seem to generate weird glyphs that only look like text. The examples they show here have perfectly legible characters and correct spelling. The difference between this and DALLE makes me suspicious / curious. I wish I could play with this model.

Still has the issue with screwing up mechanical objects. In their demo checkout the wheels on the skateboards, all over the place.

I only see the problem for the paintings. If you choose a photo it's good. Could be a problem in the source data (i.e. paintings of mechanical objects are imperfect).

Re: Imagen, a text-to-image diffusion model

#619

Post by MattSayar »

Siira wrote: post #616
interblag wrote: post #587

Earlier quoted context omitted.

I think they address some of the reasoning behind this pretty clearly in the write-up as well? > The potential risks of misuse raise concerns regarding responsible open-sourcing of code and demos. At this time we have decided not to release code or a public demo. In future work we will explore a framework for responsible externalization that balances the value of external auditing with the risks of unrestricted open-…

> harmful stereotypes and representations So we can't have this model because of ... the mere possibility of stereotypes? With this logic, humans should all die, as we certainly encode some nasty stereotypes in our brains. This level of dishonesty to not give back to the community is not unexpected at this point, but seeing apologists here is.

I think it's more that they don't want people creating NSFW images of copyrighted material. How do you even begin to protect against that litigation?

Re: Imagen, a text-to-image diffusion model

#620

Post by Firmwarrior »

octocop wrote: post #543
discmonkey wrote: post #421

For people complaining that they can't play with the model... I work at Google and I also can't play with the model :'(

is your team/division hiring?

Every tech megacorp is always hiring people who can jump through the flaming code hoops just right
Post reply on HN