Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MIT Unveils Gen AI Tool That Generates High Res Images 30 Times Faster

hothardware.com

Post reply on HN
Print view View original

41–50 of 63 posts

Re: MIT Unveils Gen AI Tool That Generates High Res Images 30 Times Faster

#41

Post by mdp2021 »

fzliu wrote: post #38
amelius wrote: post #37

Earlier quoted context omitted.

Can GANs prevent the ">10 fingers" problem?

The >10 fingers problem should be solved via higher quality data or conditioning rather than a better model architecture.

Outside of the work to study and improve available technologies ("how far can we push a hammer"),

you do not draw an individual with anomalous hands because you have an ontological model in which "humans normally have five fingers per hand".

Knowing "how the world works" is the appropriate source for subsequent expression of a representation.

Somewhere in the architecture a world model should be formed.

Re: MIT Unveils Gen AI Tool That Generates High Res Images 30 Times Faster

#42

Post by josefx »

Drakim wrote: post #40
onion2k wrote: post #39

Earlier quoted context omitted.

Are you suggesting AI generates images of people with more than 10 fingers because there are too many pictures of people with more than 10 fingers in the input data? That seems unlikely..

In some pictures not all 5 fingers on a hand are visible, so maybe it appears that humans have a variable amount of fingers?

Humans with less than 5 fingers per hand do indeed exist. How does that lead to a default of 7?

Re: MIT Unveils Gen AI Tool That Generates High Res Images 30 Times Faster

#43

Post by ilaksh »

geor9e wrote: post #14
bl0b wrote: post #3

Am I missing it or is the source code not available?

I don't think they open sourced it. Thinking back, most AI breakthrough papers I've read don't include source code unfortunately. Researchers want their names out there explaining what they did and that they did it first, but MIT might want to license the implementation IP or Adobe (sounds like their interns discovered this during a summer) lawyers might be hanging onto it for a business edge.

It's 100% Adobe.

Re: MIT Unveils Gen AI Tool That Generates High Res Images 30 Times Faster

#44

Post by ilaksh »

GaggiX wrote: post #16

There are so many similar diffusion model distillation techniques these days that it's become hard to tell the difference between them, this is probably the fourth example I've seen that uses adversarial loss to distill the model, the others being UFOgen, some other work by StabilityAI on SDXL-turbo and similar, SDXL-Lightning by ByteDance. I found this blog page that explains some of the differences: https://sander.…

This is 10 times faster than almost all of the fast image generation models.

Re: MIT Unveils Gen AI Tool That Generates High Res Images 30 Times Faster

#46

Post by spi »

mattnewton wrote: post #18
albert_e wrote: post #15

Is the text prompt input the only common thing between SD and MIT models when comparing outputs? If so I am surprised at how SIMILAR the outputs of both models look in the general layout / framing / composition of the image. How can for example both fox astronaut images have near identical backdrop of earth and earth alone on the same side of image at same apparent size. Virtually the same shade of deep blue for deep…

I believe this is similar to the Latent Consistency Modeling approach, where it’s a replacement for the “diffusion” process, not the underlying weights. Basically, they have a more efficient process for pulling images out of the weights, not necessarily a set of new weights.

The weights are different, because the model is different.

As jzbontar below mentions, the crucial point is that the random noise mask is the same. The diffusion models are trained to turn random noise to an image, and they are deterministic at that - the same noise leads to the same image.

What the authors did here was to find a smart way of training a new model able to "simulate" in a single step what diffusion achieves in many; to do so, they took many triplets of (prompt, noise, image) generated starting from random noise and a (fixed) pretrained stable diffusion checkpoint. The model is trained to replicate the results.

So, it is surprising that this works at all at creating meaningful images, but it would be _really_ surprising (i.e. probably impossible) if it generated meaningful images which were seriously different from the ones it was pretrained with!

Re: MIT Unveils Gen AI Tool That Generates High Res Images 30 Times Faster

#47

Post by poulpy123 »

fzliu wrote: post #38
amelius wrote: post #37

Earlier quoted context omitted.

Can GANs prevent the ">10 fingers" problem?

The >10 fingers problem should be solved via higher quality data or conditioning rather than a better model architecture.

it's exactly the reverse: the issues with generative AI isn't the data anymore, but the models that are not able to understand the data

Re: MIT Unveils Gen AI Tool That Generates High Res Images 30 Times Faster

#48

Post by Drakim »

josefx wrote: post #42
Drakim wrote: post #40

Earlier quoted context omitted.

In some pictures not all 5 fingers on a hand are visible, so maybe it appears that humans have a variable amount of fingers?

Humans with less than 5 fingers per hand do indeed exist. How does that lead to a default of 7?

I meant more because people are holding objects or their hand is just at an angle where all fingers are not visible in the picture.

I don't understand how neutral networks operate, but my layman's guess is that when you sometimes see hands with 5 fingers visible, sometimes 4, sometimes 3, sometimes 2, sometimes 1, and sometimes 0, then it's not immediately apparent that it means that every hand has between 5 and 0 fingers.

Think of it this way, if the AI has ever only seen houses with a maximum of 10 windows in it's entire training set, is it so unthinkable that it sometimes draws a house with 12 windows? that's a "sensible" understanding about how houses have a variable amount of windows. It just doesn't work for fingers.

I'm sure the same issue would arise if humans had other body parts that came in large quantities, but almost everything else is either 1 or 2 like the nose or eyes.

Re: MIT Unveils Gen AI Tool That Generates High Res Images 30 Times Faster

#49

Post by notresidenter »

onion2k wrote: post #39
fzliu wrote: post #38

Earlier quoted context omitted.

The >10 fingers problem should be solved via higher quality data or conditioning rather than a better model architecture.

Are you suggesting AI generates images of people with more than 10 fingers because there are too many pictures of people with more than 10 fingers in the input data? That seems unlikely..

People holding hands, people high-fiving, etc.

Re: MIT Unveils Gen AI Tool That Generates High Res Images 30 Times Faster

#50

Post by samus »

amelius wrote: post #37
fngjdflmdflg wrote: post #7

>Our method is similar to GANs in that a critic is jointly trained with the generator to minimize a divergence between the real and fake distributions, but differs in that our training does not play an adversarial game that may cause training instability, and our critic can fully leverage the weights of a pretrained diffusion model. Very glad to see GANs (or GAN-likes) coming back! Also I don't know if the examples w…

Can GANs prevent the ">10 fingers" problem?

A lot of the problems of Stable Diffusion arise because of using a text encoder that isn't up to the task to encode the meaning sufficiently precise. For example, if you mention the word "green" in the prompt, it often carries over to the whole image instead of the object that is being described. Numbers suffer from a similar problem. Rendering text is compromised by tokenization.

One problem with hands might be that they are comparatively small. The model easily gets the big picture right (head, arms, legs), but hands are so-called high-frequency details and are additionally featured in lots of different positions, which are seldom sufficiently described in the captions of the training data.

Post reply on HN