Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dall-E 2

openai.com

Post reply on HN
Print view View original

381–390 of 511 posts

Re: Dall-E 2

#381

Post by Imnimo »

astrange wrote: post #368
Imnimo wrote: post #99

Earlier quoted context omitted.

I'd say Dall-E 2 is a little more unified - they do have multiple networks, but they're trained to work together. The previous approaches I was talking about are a lot more like the microservices analogy. Someone published a model (called CLIP) that can say "how much does this image look like a sunset". Someone else published a totally different model (e.g. VQGAN) that can generate images (but with no way to provide…

VQGAN (being a "GAN") is already two networks - one Generates things, and the other is Adversarial and judges if the other network is good enough, then you train them both at once and they fight. CLIP+VQGAN generation IIRC works by replacing the adversarial network with CLIP, so it understands text prompts, then retraining it for a while towards the prompted target, then generating whatever it's learned from that. GA…

I think that in CLIP+VQGAN, the VQGAN model is frozen, and what you do is start from a random latent code, generate an image, pass it to CLIP, and the backprop through CLIP and through the VQGAN generator to figure out how you should move the latent code to make it better match the prompt. Then you just keep taking gradient ascent steps to find better and better latent codes. So it's like 'retraining', except you're 'training' the network input rather than the network weights.

Re: Dall-E 2

#382

Post by aaron695 »

recuter wrote: post #73
Imnimo wrote: post #59

I'm only part way through the paper, but what struck me as interesting so far is this: In other text-to-image algorithms I'm familiar with (the ones you'll typically see passed around as colab notebooks that people post outputs from on Twitter), the basic idea is to encode the text, and then try to make an image that maximally matches that text encoding. But this maximization often leads to artifacts - if you ask for…

What always bother me with this stuff is, well, you say one approach is more sensible than the other because the images happen to come out more pleasing. But there's no real rhyme or reason, it is a sort of alchemy. Is text encoding strictly worse or is it an artifact of the implementation? And if it is strictly worse, which is probably the case, why specifically? What is actually going on here? I can't argue that th…

No post body was provided.

Re: Dall-E 2

#383

Post by jazzyjackson »

aryamaan wrote: post #303
loufe wrote: post #278

Earlier quoted context omitted.

There are so many excellent, thought-provoking comments in this thread, but yours caught me especially. Something that came to mind immediately upon reading the release was the potential for this technology to transform literature, adding AI generated imagery to turn any novel into a visual novel as a premium way to experience the story, something akin to composing D-Box seat response to a modern movie. I was imagini…

In the same theme, I liked the comments of both of you. Another use case could be to make it easier/ automatic to create comics. You tell what the background should be, characters should be doing and the dialogues. Boom, you have a good enough comic. ----------- Reading as a medium has not evolved with technology. Creating the imagery does happen in humans' minds. It's not surprise that some people enjoy doing that (…

I can also imagine the magical nature of a child being able to make up a story (as children are wont to do) and having Dall-E here generating a picture book as they go.

Re: Dall-E 2

#384

Post by dontreact »

Imnimo wrote: post #269
nope96 wrote: post #213

Is there an 'explain it like I'm 15' for how this works? It seems like black magic. I've been a computer hobbyist since the late 1980's and this is the first time I cannot explain how a computer does what it does. Absolutely the most amazing thing I've ever seen, and I have zero clue how it works.

Here is my extremely rough ELI-15. It uses some building blocks like "train a neural network", which probably warrant explanations of their own. The system consists of a few components. First, CLIP. CLIP is essentially a pair of neural networks, one is a 'text encoder', and the other is an 'image encoder'. CLIP is trained on a giant corpus of images and corresponding captions. The image encoder takes as input an imag…

This is by far the best explanation. The other responses in this thread are either very incomplete, or incorrect.

Re: Dall-E 2

#385

Post by atleta »

chrisco255 wrote: post #348
carnitine wrote: post #75

Earlier quoted context omitted.

The tail end of programming will be the last thing to be replaced, maybe. I don’t see why CRUD apps get to hide under the umbrella of programming ultra-advanced AI.

Let me know when you can speak English to a computer and have it generate CRUD code that satisfies all engineering and design constraints. The AI will need to be dynamic enough to understand nuance, missing gaps in the requirements spec, have context on the application being built, able to suggest improvements on product design, know how to make changes through the same conversational interface, etc. Accomplishing th…

Probably not. People tend to think that tasks that make us think hard require general intelligence just because that's the tool we use to solve that problem. The AI doesn't have to be very good to be able to replace CRUD web app developers (that is, most of us).

As I see, the real challenge to solve is for it to be able to hold context and be able to communicate iteratively. Also, as you say find missing gaps. That's important. Other than that, you tell it what you want, it creates something and then you tell it to change things around. Which is, BTW, pretty similar to how it works with biological life based developers. Though as we're lazy, we like to clarify a lot of things up front (and either drive customers crazy or teach them that this is the way it works). If you have an AI that spits out code in a few minutes, it may not matter a lot.

Most of the programming jobs are indeed about making relatively simple stuff from standard components.

Re: Dall-E 2

#386

Post by drewm1980 »

andybak wrote: post #3

Preventing Harmful Generations We’ve limited the ability for DALL·E 2 to generate violent, hate, or adult images. By removing the most explicit content from the training data, we minimized DALL·E 2’s exposure to these concepts. We also used advanced techniques to prevent photorealistic generations of real individuals’ faces, including those of public figures. "And we've also closed off a huge range of potentially int…

I instinctively want to "flip the sign" on all of the automated controls they put in, just out of the morbid interest to see what comes out. The moment you have a "avoid_harm_to_humans:bool" training parameter, someone's going to set it to -1.

Their document about all the measures they took to prevent unethical use is also a document about how to use a re-implementation of their system unethically. They literally hired a "red team" of smart people to come up with the most dangerous ideas for misusing their system (or a re-implementation of it), and featured these bad ideas prominently in a very accessibly written document on their website. So many fascinating terrible ideas in there! They make a very compelling case that the technology they are developing has way more potential for societal harm than good. They had me sold at "Prompt: Park bench with happy people. + Context: Sharing as part of a disinformation campaign to contradict reports of a military operation in the park."

Re: Dall-E 2

#387

Post by Jack000 »

rahimnathwani wrote: post #379
Jack000 wrote: post #245

Earlier quoted context omitted.

With glide I think we've reached something of a plateau in terms of architecture on the "text to image generator S curve". DALL-E-2 is a very similar architecture to glide and has some notable downsides (poorer language understanding) glid-3 is a relatively small model trained by a single guy on his workstation (aka me) so it's not going to be as good. It's also not fully baked yet so ymmv, although it really depends…

Thanks! Do you happen to know how much GPU RAM I need to run glid-3 and/or the latent diffusion model, if I don't want to run on colab?

Just tried glid-3 with a batch size of one and I'm getting 4781MiB. The latent diffusion model peaks at 8403MiB

These are fp16 numbers though, you might need a recent nvidia card to run it.

Re: Dall-E 2

#388

Post by astrange »

drusepth wrote: post #351
sillysaurusx wrote: post #194

Earlier quoted context omitted.

Sam seems to be demoing something fairly close on twitter. https://twitter.com/sama/status/1511724264629678084 The solar powered ship with a propeller sailing under the golden gate bridge during sunset with dolphins jumping around was pretty impressive. https://twitter.com/sama/status/1511731259319349251 I think it's only missing the dolphins.

It should be noted that most "AI-generated" images shared (Sam's included here) are typically just a first pass, whereas most recent models also include some kind of inpainting method, where you can then mask off various parts of an image and continue to edit those specific areas until the whole image is what you're looking for. This process makes it feel a lot more like a "tool" used by artists than a simple magic b…

I'm not sure how the embeddings ("descriptions") work in DALL-E yet, but in a lot of models they're fixed-length. So there's a natural limit on how many concepts you mention in the first pass before it'll just start leaving them out.

Re: Dall-E 2

#389

Post by jupp0r »

andybak wrote: post #195
throwaway71271 wrote: post #150

Earlier quoted context omitted.

'art' comes from 'artem' which means 'skill', which is the root of 'artificial' ( https://www.etymonline.com/word/art and https://www.etymonline.com/word/artificial ) your teacher was wrong i had a friend who didnt get credit for his design work because he used photoshop instead of using pen and paper for similar reason, i still find it amazing that a teacher would say such a thing

> 'art' comes from 'artem' which means 'skill', which is the root of 'artificial' His teacher was wrong but "argument from etymology" is surely a fallacy.

Especially as I grew up in Germany where “Kunst” has different origins.

Re: Dall-E 2

#390

Post by atleta »

astrange wrote: post #376
dntrkv wrote: post #229

Earlier quoted context omitted.

I don't think anyone is in denial about this, it's just not something anyone should concern themselves with in the foreseeable future. AI that can replace a dev or designer is nowhere close to becoming a reality. Just because we have some cool demos that show some impressive capabilities in a narrow application does not mean we can extrapolate that capability to something that is many times more complex.

I do deny it. Automation does not destroy jobs even if you're impressed at how good it is at painting; see "Luddite fallacy" and "lump of labor". Claiming AIs are going to take over or destroy the world has been a basis of "AI safety" research since the 90s, but that isn't real research, it's a new religion run by Berkeley rationalists who read too many SF novels.

The assumption that automation creates (or at least does not destroy) jobs is an extrapolation from the past despite the fact that the nature of automation is constantly changing/evolving.

Also, one thing that everyone seems to ignore is that even if the number of jobs are not reduced, the skill/talent level for doing those jobs may (actually DO) increase and also, switching careers does not work for everyone. So you'll inevitably have people without a job even if it's just that the job market is shifting.

But I argue that as automation reaches jobs with higher levels of sophistication, i.e. the jobs of more skilled workers, some people will simply be left out because of their talent won't be enough to do any job that has not been automated.

Post reply on HN