Live data from Hacker News

Imagen: An AI system that creates photorealistic images from input text

imagen.research.google

81–90 of 233 posts

Re: Imagen: An AI system that creates photorealistic images from input text

#81
post #3

Not a single human face in these samples. I wonder how well it does on faces.

Saul Goodman if he was a character in Twin Peaks: https://media.discordapp.net/attachments/999426920376717513/... Generated with Midjourney Beta

Which is now Stable Diffusion under the hood, sprinkled with a little prompt parsing "magic".

Re: Imagen: An AI system that creates photorealistic images from input text

#82
post #57

Earlier quoted context omitted.

Google hasn't released squat . Google's product is vaporware and we shouldn't afford them any airtime until they release something usable. They're just trying to butt in and get press off of the backs of the teams actually working in the open, and that's super lame. Release your model, Google, or stop bragging and talking over the others here. You're greedily sucking oxygen out of the conversation, and as a trillion…

I kind of get the sentiment about openness but I think it's way more nuanced than you are making out. There are very good reasons for withholding SOTA models, primarily from the info hazard angle and avoiding escalating the capabilities race which is basically the biggest risk we have right now. Google / Deepmind have actually made some good decisions to try and slow down the race (such as waiting to publish).

Capabilities race, seriously? This is not nuclear warfare my guy. It's mathematics.

Re: Imagen: An AI system that creates photorealistic images from input text

#83

Earlier quoted context omitted.

Is 3d a different problem, or a similar one but considerably harder? I'd expect the data encoding (vertices vs pixels) to change a bit about it but I'm not familiar enough to know.

Pixel values are discrete (length x width x r256 x g256 x b256) and vertex values are continuous, so that is one major difference. Secondly, there's vastly more labeled image data in the world than 3D data, so creating a CLMP (contrastive language and mesh pairing) model is harder. It's very late but I may be able to give a much better answer on more of the nuances of 3D generation tomorrow.

Would voxels be easier than vertex-based meshes?

I can imagine you'd have the problem of stray floating voxels then, which isn't as noticeable when it happens with 2D pixels.

Re: Imagen: An AI system that creates photorealistic images from input text

#84
post #74

Every single time I see an article about a new AI model that has a section called "societal impact" I know immediately they are not releasing the model, the training set, nothing... It seems to be the kind of bullshit statement that those companies put in place of "we paid $500k training this model and we're not giving it for free to anyone".

Fortunately we have Stability.AI and they release their image generation model already. Hopefully they'll follow with other projects too.

https://stability.ai/blog/stable-diffusion-public-release

Re: Imagen: An AI system that creates photorealistic images from input text

#85

These tools are amazing for prototyping. I had an idea for a promotional poster, and seeing my idea just by writing it felt like magic. The generated image had too many artifacts to use, but gave me a guideline to follow when creating the real thing in Pixlr. AI content generation (text, image, source code, video, music) will be a huge boon for prototyping where applied judiciously.

This was the first thing I tried with DALL-E. Took some photos of my house where I'm renovating, wiped out the construction debris and told it to fill it in with what I wanted.

It worked okay - one issue was DALL-E wants to keep "style" consistent so any stray bit of debris greatly affected the interpretation, but I did in fact get 1 design idea out of it which changed how I think we'll do a bit of it.

These things in many ways are just extremely enhanced search tools - "describe what you want to see"

Re: Imagen: An AI system that creates photorealistic images from input text

#86

Is there an AI that creates realistic text from photos?

Look into image-to-text or CLIP captioning. There are several tools out there, although similar to running text-to-image tools locally, may require some time to setup via conda. The easiest way that I know of (currently) would be install Visions of Chaos. You will still need to follow the TensorFlow setup guide[0] which will take 1-2 hours, then, running Visions of Chaos > Mode > Machine Learning will download 3-400GB of models in order to be able to run all the available tools they have integrated, even if you aren't interested in most of them. The setup may require several retries and several hours due to some servers being extremely slow (or overloaded).

EDIT: Just found this[1] as well, though setup might also be a pain.

[0] https://softology.pro/tutorials/tensorflow/tensorflow.htm [1] https://replicate.com/methexis-inc/img2prompt

Re: Imagen: An AI system that creates photorealistic images from input text

#87
post #18

Earlier quoted context omitted.

Someone on the midjourney discord is making a comic with images generated by the bot. Another person is making a Magic The Gathering card pack with generated art (it looks good too!). People are already using it for stock photos One murky area we're still far away from but I'm curious to follow the developments on: AI-generated movies. Once generated clips gets good, what if some movie buff can just generate a movie,…

How do they get consistent looks? I mean if you are using it for a comic book your characters should look the same throughout the story. From my experience, the results between prompts are completely different even with minor variations.

If the seed remains the same, the difference between similar prompts is actually continuous.

Here's a slew of images (1 through 5) I generated all from the same seed and same prompt sans a word or two: https://www.instagram.com/p/Chg60Fou6xB/

Re: Imagen: An AI system that creates photorealistic images from input text

#89

There are some obvious mistakes in tools like this. Such as: Human faces are wrong, writing is usually scrambled, fingers look weird etc... Do you know if we need to have a major breakthrough similar to what happened 6 months ago to fix this or could these be fixed with incremental improvements in current techniques / datasets?

The writing issue demonstrably has been solved without any breakthroughs by simply making a larger model (dall-E 2 vs the publicly available dall-e), the same appears to be for other main issues as well - it's just that the publicly available versions are based on smaller/weaker models than the state of art because they're significantly cheaper to run.

Also, I seem to recall that at least some models deliberately harmed generation of human faces (e.g. by selection of training data) to draw away attention from the deepfake/fakenews usecases and the related ethical,political and PR issues; I would assume that if any of them wanted to actually try and make specifically faces look good, that would be purely a matter of some engineering work without any breakthroughs needed - I mean, we have evidence from face-specific models that the same technical architecture can do decent faces.

Re: Imagen: An AI system that creates photorealistic images from input text

#90
Coming from Google? No thanks. If we are already being screwed by simple services like YouTube, I cannot imagine how screwed up we'll be when we become addicted to services that rely on the research posted on the article. Coming from Google, it's probably only good for Google, not for its potential customers/users/slaves.
Post reply on HN