Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video

blog.comfy.org

Post reply on HN
Print view View original

51–60 of 100 posts

Re: MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video

#51

Post by liuliu »

yorwba wrote: post #28
embedding-shape wrote: post #15

> We found that the model's modulation weights (~40% of the total parameters) could be pruned and replaced with a functionally equivalent lookup table, dramatically shrinking the memory footprint with no loss in output quality. Is this a common approach to reducing weights with "no loss in output quality", assuming this is true? Seems almost too simple to work. If this is doable, would this be applicable to LLMs as w…

"Modulation weights" here refers to weights used to adjust layer normalization depending on the task (adaLN). General-purpose LLMs generally don't have those in the first place.

One thing similar would be projecting both the head.weight and the final LLM activations into a smaller vector space, since that is basically just cosine similarity ranking step (so that would reduce the head.weight size). But again, it must be tried many times and just not working as well. LLM space is pretty saturated with tricks.

Re: MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video

#52

Post by xienze »

liuliu wrote: post #46
embedding-shape wrote: post #15

> We found that the model's modulation weights (~40% of the total parameters) could be pruned and replaced with a functionally equivalent lookup table, dramatically shrinking the memory footprint with no loss in output quality. Is this a common approach to reducing weights with "no loss in output quality", assuming this is true? Seems almost too simple to work. If this is doable, would this be applicable to LLMs as w…

It is a well-known trick, given that the timestep is between 0 to 1, you can slicing them at any resolution (1000, or 10000, give or take), and then keep a look-up table for modulation scale / bias etc for each. It is quite different from quantization and it is indeed lossless. It is also only applicable to diffusion models as only these operates at per-timestep.

So... why didn't the model ship this way to begin with? They just wanted to waste VRAM for fun?

Re: MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video

#53

Post by liuliu »

xienze wrote: post #52
liuliu wrote: post #46

Earlier quoted context omitted.

It is a well-known trick, given that the timestep is between 0 to 1, you can slicing them at any resolution (1000, or 10000, give or take), and then keep a look-up table for modulation scale / bias etc for each. It is quite different from quantization and it is indeed lossless. It is also only applicable to diffusion models as only these operates at per-timestep.

So... why didn't the model ship this way to begin with? They just wanted to waste VRAM for fun?

They ship a complete checkpoint for easily management (inference & training) in their own infrastructure. Moving to a LUT would make training on these layers impossible. BTW, these are not useful for lightweight fine-tuning, but might still be useful if you do serious post-training work.

Of course, these are also not an issue for things like FLUX.2 which adopts DiT-Air arch, that doesn't have this wasted space issue.

Re: MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video

#54

Post by jareklupinski »

sheesdev wrote: post #7

The mouse render is surprisingly good. Several of those clips stood out to be a pretty big leap in terms of current SOTA models. The only one that looks "off" is the beverage ad video during the can opening clip, it still has that "AI smoothening" effect. Good thing this can be done pretty well using traditional rendering. I feel like for a good while now we'll transition into a process that uses traditional "close-u…

right before that, there's the part with the person hiking up the dish and 'breathing', and the clouds of water vapor coming out of their mouth don't line up with their breaths

devs pls fix

Re: MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video

#55

Post by doctorpangloss »

xienze wrote: post #52
liuliu wrote: post #46

Earlier quoted context omitted.

It is a well-known trick, given that the timestep is between 0 to 1, you can slicing them at any resolution (1000, or 10000, give or take), and then keep a look-up table for modulation scale / bias etc for each. It is quite different from quantization and it is indeed lossless. It is also only applicable to diffusion models as only these operates at per-timestep.

So... why didn't the model ship this way to begin with? They just wanted to waste VRAM for fun?

It may or may not be true. The people who made this modification and the other commenters didn't do anything rigorous to verify what they did. They just eyeball it. They could very well make some other error - this has happened frequently - that developing on prod, not knowing what they are doing, has and hopefully will again solve.

Re: MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video

#56

Post by teaearlgraycold »

embedding-shape wrote: post #25
Maxious wrote: post #14

Earlier quoted context omitted.

FWIW on a 5080 16GB it takes 3 minutes for 10 seconds 480p video (the mouse video workflow with length changed from 5 seconds to 10 seconds)

FWIW on a 6000 Pro it takes 68 seconds for 10 seconds 480p video, (cold) same demo workflow as you used. Set "megapixels" to 2.0 (1920x1088) and same video seems to take 5+ minutes, not sure if everything is right/correct at the moment. As far as I can tell, the current ComfyUI nodes don't even do compilation, and I haven't looked into what attention mechanism they're using, but I'm sure with time these durations wil…

Waiting for the “FWIW on a B200” reply

Re: MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video

#57

Post by wincy »

embedding-shape wrote: post #25
Maxious wrote: post #14

Earlier quoted context omitted.

FWIW on a 5080 16GB it takes 3 minutes for 10 seconds 480p video (the mouse video workflow with length changed from 5 seconds to 10 seconds)

FWIW on a 6000 Pro it takes 68 seconds for 10 seconds 480p video, (cold) same demo workflow as you used. Set "megapixels" to 2.0 (1920x1088) and same video seems to take 5+ minutes, not sure if everything is right/correct at the moment. As far as I can tell, the current ComfyUI nodes don't even do compilation, and I haven't looked into what attention mechanism they're using, but I'm sure with time these durations wil…

What’s the vram usage? I’ve got a 5090 so as long as I can fit it in memory the gen times should be roughly equivalent ~10%

Re: MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video

#58

Post by ranyume »

Mashimo wrote: post #6

> The result gives a total memory footprint reduced by 66%, from 123.6 GB in full precision to 42.5 GB with the smallest models variants. Combining this with our dynamic VRAM offloading enables a next-generation 2K video model to run locally on a GPU like the RTX 3060. Pretty cool. But assuming you have a 16GB 3060, how long would it take to generate a 15 second clip?

Around 10 minutes.

Re: MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video

#60

Post by halJordan »

teaearlgraycold wrote: post #56
embedding-shape wrote: post #25

Earlier quoted context omitted.

FWIW on a 6000 Pro it takes 68 seconds for 10 seconds 480p video, (cold) same demo workflow as you used. Set "megapixels" to 2.0 (1920x1088) and same video seems to take 5+ minutes, not sure if everything is right/correct at the moment. As far as I can tell, the current ComfyUI nodes don't even do compilation, and I haven't looked into what attention mechanism they're using, but I'm sure with time these durations wil…

Waiting for the “FWIW on a B200” reply

Runpod waiting for you
Post reply on HN