Live data from Hacker News

Dall-E 2

openai.com

191–200 of 511 posts

Re: Dall-E 2

#191
post #139
post #59

I'm only part way through the paper, but what struck me as interesting so far is this: In other text-to-image algorithms I'm familiar with (the ones you'll typically see passed around as colab notebooks that people post outputs from on Twitter), the basic idea is to encode the text, and then try to make an image that maximally matches that text encoding. But this maximization often leads to artifacts - if you ask for…

Do you think some of these techniques could be slightly modified, and applied to DNA sequences?

Maybe very very short (single-gene) sequences. The thing with DNA is it's the product of evolution. The DNA guides the synthesis of proteins, then the proteins fold into a 3D shape, and they interact with chemicals in their environment based on their shape.

In the context of a living being, different genes interact with each other as well. For example, you have certain cells that secrete hormones (many genes needed to do that), then you have genes that encode for hormone receptors, and those receptors trigger other actions encoded by other genes. There's probably too much complexity to ask an AI system to synthesize the entire genetic code for a living being. That would be kind of like if I asked you to draw the exact blueprints for a fighter get, and write all the code, and synthesize all the hardware all at once, and you only get one shot. You would likely fail to predict some of the interactions and the resulting system wouldn't work. You could only achieve this through an iterative process that would involve years of extensive testing.

Could you use a deep learning system to synthesize genetic code? Maybe just single genes that do fairly basic things, and you would need a massive dataset. Hard to say what that would look like. Is it really enough to textually describe what a gene does?

Re: Dall-E 2

#192

Very cool stuff. For me, the most interesting was the ability to take a piece of art and generate variations of it. Have a favorite painter? Here's 10,000 new paintings like theirs.

I was just thinking the same thing, how awesome would it be to be able to use this in conjunction with the Samsung frame in art gallery mode and have it just generate novel paintings in the style of your favorite painters.

Re: Dall-E 2

#193
post #5

Some freely available models GLID-3: https://colab.research.google.com/drive/1x4p2PokZ3XznBn35Q5B... and a new Latent Diffusion notebook: https://colab.research.google.com/github/multimodalart/laten... have both appeared recently and are getting remarkably close to the original Dall-E (maybe better as I can't test the real thing...) So - this was pretty good timing if OpenAI want to appear to be ahead of the pack. Of…

How do you run such a Google Colab thing? I don't see a run button? On.. maybe "Runtime -> Run All" from the menu ... Shows me a spinning circle around "Download model" ... 26% ... Fascinating, that Google offers you a computer in the cloud for free .. Now it is running the model. Wow, I'm curious .. Ha, it worked! Nothing compared to the images in the Dall-E 2 article but still impressive.

Google is a company with a lot of spare VMs and GPUs.

However, the free GPU is now a K80 which is obsolete and barely sufficient for running these types of models.

Re: Dall-E 2

#194
post #17

This is mind blowing. I was not expecting the sketch style images to actually look like sketches. Style transfer based sketches never look like sketches. This and the current AI generated art scene makes it looks like that artwork is now a "solved" problem. See AI generated art on twitter etc. There is a strong relation between the prompt and the generated images but just like GPT-3, it fails to fully understand what…

I would caution that artwork is only 'solved' with relatively simple text prompts. To create a novel painting with a precise mix of elements that would take a paragraph or more to explain is still tough, though DALL-E 2 does seem like a big step towards that.

Sam seems to be demoing something fairly close on twitter. https://twitter.com/sama/status/1511724264629678084

The solar powered ship with a propeller sailing under the golden gate bridge during sunset with dolphins jumping around was pretty impressive. https://twitter.com/sama/status/1511731259319349251

I think it's only missing the dolphins.

Re: Dall-E 2

#195
post #83

Earlier quoted context omitted.

This reminds me of an art class in high school in the early 2000s where I handed in a printout of a 3d generated image (painstakingly modeled and rendered in software over the whole weekend by me) and the teacher looked at me and told me that's not art because it's "computer generated" and I didn't "even use my hands" to make it. Even as a teenager, the idea that art is defined by how it's made versus it being a way…

'art' comes from 'artem' which means 'skill', which is the root of 'artificial' ( https://www.etymonline.com/word/art and https://www.etymonline.com/word/artificial ) your teacher was wrong i had a friend who didnt get credit for his design work because he used photoshop instead of using pen and paper for similar reason, i still find it amazing that a teacher would say such a thing

> 'art' comes from 'artem' which means 'skill', which is the root of 'artificial'

His teacher was wrong but "argument from etymology" is surely a fallacy.

Re: Dall-E 2

#196
It's becoming clear that efficient work in the future will hinge upon one's ability to accurately describe what one wants. Unpacking that -- a large piece is the ability to understand all the possible "pitfalls" and "misunderstandings" that could happen on the way to a shared understanding.

While technical work will always have a place -- I think that much creative work will become more like the management of a team of highly-skilled, niche workers -- with all the frustrations, joys, and surprises that entails.

Re: Dall-E 2

#197
post #157
post #90

Earlier quoted context omitted.

Yeah, I mean you're right that ultimately the proof is in the pudding. But I do think we could have guessed that this sort of approach would be better (at least at a high level - I'm not claiming I could have predicted all the technical details!). The previous approaches were sort of the best that people could do without access to the training data and resources - you had a pretrained CLIP encoder that could tell you…

I gotta be missing something here, because wasn’t “teaching a three year old to paint” (where the three year old is DALLE) the original objective in the first place? So if we’ve reduced the problem to that, it seems we’re back where we started. What’s the difference?

I meant to say that Dall-E 2's approach is closer to "teaching a three year old to paint" than the alternative methods. Instead of trying to maximize agreement to a text embedding like other methods, Dall-E 2 first predicts an image embedding (very roughly analogous to envisioning what you're going to draw before you start laying down paint), and then the decoder knows how to go from an embedding to an image (very roughly analogous to "knowing how to paint"). This is in contrast to approaches which operate by repeatedly querying "does this look like the text prompt?" as they refine the image (roughly analogous to not really knowing how to paint, but having a critic who tells you if you're getting warmer or colder).

Re: Dall-E 2

#200

I don’t want to dismiss this new model and achievements but we are getting to the point where I feel like what we saw in the open source versus close source systems we see in new ml models another one is forming for open and closed models. I think that larger and larger models will have disclaimers either restricting you from using it commercially (a great deal of academics and NVIDIA models are doing this. And OpenA…

They're pretty strict about usage:

- https://github.com/openai/dalle-2-preview/blob/main/system-c...

- https://github.com/openai/dalle-2-preview/blob/main/system-c...

Post reply on HN