Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dall-E 2

openai.com

Post reply on HN
Print view View original

231–240 of 511 posts

Re: Dall-E 2

#231

Post by joshcryer »

ordu wrote: post #204

Dall-E 2 seems to be incapable to catch the essence of the art. I'm not really surprised by it, I'd be surprised a lot if it could. But nevertheless: if you looked in the eye of a Girl With A Pearl Earring[1], you'd be forced to stop and to think what does she have on her mind right now. Or may be you had some other question in your mind, but it really stops people to think. But none of Dall-E interpretations have th…

What do you think of the third to last image of the Girl With A Pearl Earring that DALL-E 2 created? I find it more compelling than the original with how her face is deeply cast in shadow. There's still that original 'essence' of the glint in her eye. But her earring is a bell. As if the AI is sending a message that what if the bell were to ring?

Re: Dall-E 2

#232

Post by recuter »

mrandish wrote: post #220
recuter wrote: post #73

Earlier quoted context omitted.

What always bother me with this stuff is, well, you say one approach is more sensible than the other because the images happen to come out more pleasing. But there's no real rhyme or reason, it is a sort of alchemy. Is text encoding strictly worse or is it an artifact of the implementation? And if it is strictly worse, which is probably the case, why specifically? What is actually going on here? I can't argue that th…

> This is a fantastic and time saving photoshop plugin. But what do we have really? Stock photography sales are in the many billions of dollars per year and custom commissioned photography is larger still. That's a pretty seriously sized ready-made market. > But if doesn't spit up exactly what you want can't edit it further. I suspect there's a big startup opportunity in pioneering an easy-to-use interface allowing u…

Certainly food for thought.

Say I'm looking for photography of real events and places, like a royal weeding or a volcano erupting does this help me? Of specific places and architectural features? Of a protest?

You're suggesting clipart on steroids: https://thispersondoesnotexist.com

I think if I was istockphoto.com I'd be a little worried, but that is microstock photography. I'm not sure that is worth billions. In fact I know it isn't.

Besides once this tech is wildly available if anything it devalues this sort of thing further closer to $0.

It would probably augment existing processes rather than replace them completely.

If you are doing a photoshoot for a banana stand with a human model with characteristics x,y,z you're still going to get a human from an agency or craigslist to pose. If suddenly the client informs you that they needed human a,b,c instead maybe one of these forthcoming tools will let you swap that out faster. You'd upload your photoshoot and an example or two of the type of human model you wished you had retroactively and it would fix it up faster than an intern.

Cool.

Re: Dall-E 2

#234

Post by jdrc »

zitterbewegung wrote: post #182

I don’t want to dismiss this new model and achievements but we are getting to the point where I feel like what we saw in the open source versus close source systems we see in new ml models another one is forming for open and closed models. I think that larger and larger models will have disclaimers either restricting you from using it commercially (a great deal of academics and NVIDIA models are doing this. And OpenA…

It should be possible to create open source versions, researchers will find a way if something is cool enough

Re: Dall-E 2

#235

Post by dash2 »

recuter wrote: post #123
Imnimo wrote: post #103

Earlier quoted context omitted.

This is actual image generation - the 'decoder' takes as input a latent code (representing the encoding of the text query), and synthesizes an image. It's not compositing or querying a reference library. The only time that real images enter the process is during training - after that, it's just the network weights.

It is compositing as final step. I understand that the Kuala it is compositing may have been a previously un-existent Kuala that it synthesized from a library of previously tagged Kuala images... that's cool, but what is the difference really from just plucking one of the pre-existing Kualas into the scene? The difference is just that it makes the compositing easier. If you don't have a pre-existing image that would…

>And if the basketball background has wind blowing leaves in one direction the Kuala fur won't match, it will look like the training set fur. The puddle won't reflect it.

If you read the article, it gives examples that do exactly this. For example, adding a flamingo shows the flamingo reflected in a pool. Adding a corgi at different locations in a photo of an art gallery shows it in picture style when it's added to a picture, then in photorealistic style when it's on the ground.

Re: Dall-E 2

#236

Post by rileyphone »

recuter wrote: post #73
Imnimo wrote: post #59

I'm only part way through the paper, but what struck me as interesting so far is this: In other text-to-image algorithms I'm familiar with (the ones you'll typically see passed around as colab notebooks that people post outputs from on Twitter), the basic idea is to encode the text, and then try to make an image that maximally matches that text encoding. But this maximization often leads to artifacts - if you ask for…

What always bother me with this stuff is, well, you say one approach is more sensible than the other because the images happen to come out more pleasing. But there's no real rhyme or reason, it is a sort of alchemy. Is text encoding strictly worse or is it an artifact of the implementation? And if it is strictly worse, which is probably the case, why specifically? What is actually going on here? I can't argue that th…

It would be interesting to see more attempts to “reverse engineer” ML models like in https://distill.pub/2020/circuits/curve-circuits - maybe even with a ML model of its own?

Re: Dall-E 2

#237

Post by recuter »

Jack000 wrote: post #219
recuter wrote: post #165

Earlier quoted context omitted.

https://cdn.openai.com/papers/dall-e-2.pdf > Given an image x, we can obtain its CLIP image embedding zi and then use our decoder to “invert” zi, producing new images that we call variations of our input. .. It is also possible to combine two images for variations. To do so, we perform spherical interpolation of their CLIP embeddings zi and zj to obtain intermediate zθ = slerp(zi, zj , θ), and produce variations of z…

The first quote is talking about prompting the model with images instead of text. The second quote is using "mix up" in the sense that the model is confused about the prompt, not that it mixes up existing images. ML models can output training data verbatim if they over-fit, but a well trained model does extrapolate to novel inputs. You could say that this model doesn't know that images are 2d representations of a lar…

The model is "confused about the prompt" because it has no concept of a scene or of (some sort of) reality.

If we task "Kuala dunking basketball" to a human and present them with two images, one of a Kuala climbing a tree and another of a basketball player dunking - the human would cut out the foreground (Human, Kuala) from the background (basketball court, forest) and swap them places easily.

The laborious part would be to match the shadows and angles in the new image. This requires skill and effort.

Dall-E would conjure up an entirely novel image from scratch, dodging this bit. It blended the concepts instead, great.

But it does not understand what a basketball court actually is, or why the Kuala would reflect in a puddle. Or why and how this new Kuala might look different in these circumstances from previous examples of Kualas that it knows about.

The human dunker and the kuala dunker are not truly interchangeable. :)

Re: Dall-E 2

#238

Post by moyix »

recuter wrote: post #73
Imnimo wrote: post #59

I'm only part way through the paper, but what struck me as interesting so far is this: In other text-to-image algorithms I'm familiar with (the ones you'll typically see passed around as colab notebooks that people post outputs from on Twitter), the basic idea is to encode the text, and then try to make an image that maximally matches that text encoding. But this maximization often leads to artifacts - if you ask for…

What always bother me with this stuff is, well, you say one approach is more sensible than the other because the images happen to come out more pleasing. But there's no real rhyme or reason, it is a sort of alchemy. Is text encoding strictly worse or is it an artifact of the implementation? And if it is strictly worse, which is probably the case, why specifically? What is actually going on here? I can't argue that th…

> But if doesn't spit up exactly what you want can't edit it further.

Why? You can tweak the prompt, change parameters, or even use the actual "edit" capability that they demo in the post.

Re: Dall-E 2

#239

Post by hackinthebochs »

dntrkv wrote: post #229
0F wrote: post #144

Earlier quoted context omitted.

Literally everyone on this website is in denial. They all approach it by asking which fields will be safe. No field is safe. “But it’s not going to happen for a long time.” Climate deniers say the same thing and you think they should be wearing the dunce hat? The average person complains bitterly about climate deniers who say that it’s “my grandkids problem lol” but when I corner the average person into admitting AI…

I don't think anyone is in denial about this, it's just not something anyone should concern themselves with in the foreseeable future. AI that can replace a dev or designer is nowhere close to becoming a reality. Just because we have some cool demos that show some impressive capabilities in a narrow application does not mean we can extrapolate that capability to something that is many times more complex.

What does nowhere close mean to you? 10 years? 50 years?

Re: Dall-E 2

#240

Post by awinter-py »

They're using training set restriction and prompt engineering to control its output

> By removing the most explicit content from the training data, we minimized DALL·E 2’s exposure to these concepts

> We won’t generate images if our filters identify text prompts and image uploads that may violate our policies

The 'how to prevent superintelligences from eating us' crowd should be taking note: this may be how we regulate creatures larger than ourselves in the future

And even how we regulate the ethics of non-conscious group minds like big companies

Post reply on HN