https://github.com/lucidrains/imagen-pytorch
Is this a joke?
No
Imagen, a text-to-image diffusion model
351–360 of 661 posts
Re: Imagen, a text-to-image diffusion model
#352Hey I also wrote a neural net that generates perfect images. Here's a static site about it. With images it definitely generated! Can you use it? Is there a source? Hah, of course not, because ethics!
Re: Imagen, a text-to-image diffusion model
#353Interesting to me that this one can draw legible text. DALLE models seem to generate weird glyphs that only look like text. The examples they show here have perfectly legible characters and correct spelling. The difference between this and DALLE makes me suspicious / curious. I wish I could play with this model.
Still has the issue with screwing up mechanical objects. In their demo checkout the wheels on the skateboards, all over the place.
Re: Imagen, a text-to-image diffusion model
#354Earlier quoted context omitted.
Yep, that's the hard problem Google is not comfortable releasing the API to this until they have it solved.
But why is it a problem? The AI is just a mirror showing us ourselves. That’s a good thing. How does it help anyone to make an AI that presents a fake world so that we can pretend that we live in a world that we actually don’t? Disassociation from reality is more dangerous than bias.
> The AI is just a mirror showing us ourselves.
That's one hypothesis.
Re: Imagen, a text-to-image diffusion model
#355Earlier quoted context omitted.
I agree that it’s still a jump ball in a rapidly moving field, I was saying Google is far ahead, not that they’ve won. And I don’t think whatever iteration of PaLM was cooking at the time GPT-3 started getting press would have looked to shabby. I think Google crushed OpenAI on both GPT and DALL-E in short order because OpenAI published twice and someone had had enough.
That’s pretty speculative and dubious (the holding back part) given the heavy bias to publication culture at Google Research and DeepMind. OpenAI has hardly been “crushed” here; PaLM and Imagen are solid, incremental advances, but given what came before them, not Earth-shattering. If I were going to cite evidence for Alphabet’s “supremacy” in AI, I would’ve picked something more novel and surprising such as AlphaFold…
I want to be clear, all of this stuff is fascinating, expensive, and difficult. With the possible exception of a few trailer-park weirdos like me, it basically takes a PhD to even stay on top of the field, and you clearly know your stuff.
And to be equally clear, I have no inside baseball on how Brain/DM choose when to publish. I have some watercooler chat on the friendly but serious rivalry between those groups, but that’s about it.
I’m looking from the outside in at OpenAI getting all the press and attention, which sounds superficial but sooner or later turns into actual hires of actual star-bound post docs, and Google laying a little low for a few years.
Then we get Gato, Imagen, and PaLM in the space of like what, 2 months?
Clearly I’m speculating that someone pulled the trigger, but I don’t think it’s like, absurd.
Re: Imagen, a text-to-image diffusion model
#356I'm curious why all of these tools seem to be almost tailored toward making meme images? The kind of early 2010's, over the top description of something that's ridiculous
My hunch is that they aren't tailored toward ridiculous images exactly, but if they demonstrated "a woman sitting in a chair reading", it would be really hard to tell if the result was a small modification of an image in the training data. If they demonstrate "A snake made out of corn", I have less concern about the model having a very close training example.
Re: Imagen, a text-to-image diffusion model
#357Earlier quoted context omitted.
> I mean, from my perspective, the skill in these (and DALL-E's) image reproductions is truly astonishing. A basic part of it is that neural networks combine learning and memorizing fluidly inside them, and these networks are really really big, so they can memorize stuff good. So when you see it reproduce a Shiba Inu well, don’t think of it as “the model understands Shiba Inus”. Think of it as making a collage out of…
To be clear, I understand the general techniques about (a) how diffusion models can be used to upsample images and generate more photorealistic (or even "cartoon realistic") results and (b) I understand how they can do basic matching of "someone typed in Shiba Inu, look for images of Shiba Inus". What I don't understand is how they do the composition . E.g. for "A giant cobra snake on a farm. The snake is made out of…
a) Diffusion is not just used to upsample images but also to create them.
b) It has seen images with descriptions of "corn," "cobra," "farm," and it has seen images of "A made out of B" and "C on a D." To generate a high-scoring image, it has to make something that scores well on all of them put together.
Re: Imagen, a text-to-image diffusion model
#358Earlier quoted context omitted.
Looks like no, "The potential risks of misuse raise concerns regarding responsible open-sourcing of code and demos. At this time we have decided not to release code or a public demo. In future work we will explore a framework for responsible externalization that balances the value of external auditing with the risks of unrestricted open-access."
> the risks of unrestricted open-access What exactly is the risk?
"Make a photograph of Joe Biden in a hotel room bed with Kim Jong-un."
Simply the ease at which people are going to be able to make extremely-realistic game photographs is going to do some damage to the world. It's inevitable, but it might be good to postpone it.
Re: Imagen, a text-to-image diffusion model
#359Earlier quoted context omitted.
See section 6 titled “Conclusions, Limitations and Societal Impact” in the research paper: https://gweb-research-imagen.appspot.com/paper.pdf One quote: > “On the other hand, generative methods can be leveraged for malicious purposes, including harassment and misinformation spread [20], and raise many concerns regarding social and cultural exclusion and bias [67, 62, 68]”
But do we trust that those who do have access won't be using it for "malicious purposes" (which they might not think is malicious, but perhaps it is to those who don't have access)?
[deleted]
Re: Imagen, a text-to-image diffusion model
#360As someone who has a layman's understanding of neural networks, and who did some neural network programming ~20 years ago before the real explosion of the field, can someone point to some resources where I can get a better understanding about how this magic works? I mean, from my perspective, the skill in these (and DALL-E's) image reproductions is truly astonishing. Just looking for more information about how the so…
> I mean, from my perspective, the skill in these (and DALL-E's) image reproductions is truly astonishing. A basic part of it is that neural networks combine learning and memorizing fluidly inside them, and these networks are really really big, so they can memorize stuff good. So when you see it reproduce a Shiba Inu well, don’t think of it as “the model understands Shiba Inus”. Think of it as making a collage out of…
> these networks are really really big, so they can memorize stuff good.
People tend to really underestimate just how big these models are. Of course these models aren't simply "really really big" MLPs, but the cleverness of the techniques used to build them is only useful at insanely large scale.
I do find these models impressive as examples of "here's what the limit of insane amounts of data, insane amounts of compute can achieve with some matrix multiplication". But at the same time, that's all they are.
What saddens me about the rise of deep neural networks is it is really is the end of the era of true hackers. You can't reproduce this at home. You can't afford to reproduce this one in the cloud with any reasonable amount of funding. If you want to build this stuff your best bet is to go to top tier school, make the right connections and get hired by a mega-corp.
But the real tragedy here is that the output of this is honestly only interesting it if it's the work of some hacker fiddling around in their spare time. A couple of friend hacking in their garage making images of raccoon painting is pretty cool. One of the most powerful, well funded, owners of the likely the most compute resources on the planet doing this as their crowning achievement in AI... is depressing.