Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Consistency diffusion language models: Up to 14x faster, no quality loss

together.ai

Post reply on HN
Print view View original

51–60 of 101 posts

Re: Consistency diffusion language models: Up to 14x faster, no quality loss

#51

Post by noosphr »

block_dagger wrote: post #36
5o1ecist wrote: post #29

Earlier quoted context omitted.

[flagged]

Doesn’t the widely accepted Bitter Lesson say the exact opposite about specialized models vs generalized?

The corollary to the bitter lesson is that in any market meaningful time scale a human crafted solution will outperform one which relies on compute and data. It's only on time scales over 5 years that your bespoke solution will be over taken. By which point you can hand craft a new system which uses the brute force model as part of it.

Repeat ad-nauseam.

I wish the people who quote the blog post actually read it.

Re: Consistency diffusion language models: Up to 14x faster, no quality loss

#52

Post by 5o1ecist »

stavros wrote: post #32
5o1ecist wrote: post #29

Earlier quoted context omitted.

[flagged]

Why don't we need them? If I need to run a hundred small models to get a given level of quality, what's the difference to me between that and running one large model?

[flagged]

Re: Consistency diffusion language models: Up to 14x faster, no quality loss

#54

Post by adgjlsfhk1 »

irthomasthomas wrote: post #43
wongarsu wrote: post #28

Earlier quoted context omitted.

Notice how all the major AI companies (at least the ones that don't do open releases) stopped telling us how many parameters their models have. Parameter count was used as a measure for how great the proprietary models were until GPT3, then it suddenly stopped. And how inference prices have come down a lot, despite increasing pressure to make money. Opus 4.6 is $25/MTok, Opus 4.1 was $75/MTok, the same as Opus 4 and…

You're hitting on something really important that barely gets discussed. For instance, notice how opus 4.5's speed essentially doubled, bringing it right in line with the speed of sonnet 4.5? (sonnet 4.6 got a speed bump too, though closer to 25%). It was the very first thing I noticed: it looks suspiciously like they just rebranded sonnet as opus and raised the price. I don't know why more people aren't talking abou…

opus 4.6 was going to be sonet 5 up until week of release. The price bump is even bigger than you realize because they don't let you run opus 4.6 at full speed unless you pay them an extra 10x for the new "fast mode"

Re: Consistency diffusion language models: Up to 14x faster, no quality loss

#55

Post by [deleted] »

somewhereoutth wrote: post #34
5o1ecist wrote: post #29

Earlier quoted context omitted.

[flagged]

I'd suggest that a measure like 'density[1]/parameter' as you put it will asymptotically rise to a hard theoretical limit (that probably isn't much higher than what we have already). So quite unlike Moore's Law.

[deleted]

Re: Consistency diffusion language models: Up to 14x faster, no quality loss

#56

Post by red75prime »

5o1ecist wrote: post #29
wongarsu wrote: post #28

Earlier quoted context omitted.

Notice how all the major AI companies (at least the ones that don't do open releases) stopped telling us how many parameters their models have. Parameter count was used as a measure for how great the proprietary models were until GPT3, then it suddenly stopped. And how inference prices have come down a lot, despite increasing pressure to make money. Opus 4.6 is $25/MTok, Opus 4.1 was $75/MTok, the same as Opus 4 and…

[flagged]

Obviously, there’s a limit to how much you can squeeze into a single parameter. I guess the low-hanging fruit will be picked up soon, and scaling will continue with algorithmic improvements in training, like [1], to keep the training compute feasible.

I take "you can't have human-level intelligence without roughly the same number of parameters (hundreds of trillions)" as a null hypothesis: true until proven otherwise.

[1] https://arxiv.org/html/2602.15322v1

Re: Consistency diffusion language models: Up to 14x faster, no quality loss

#57

Post by Someone »

5o1ecist wrote: post #52
stavros wrote: post #32

Earlier quoted context omitted.

Why don't we need them? If I need to run a hundred small models to get a given level of quality, what's the difference to me between that and running one large model?

[flagged]

> There is not one person on the planet, who wouldn't prefer a doctor who is deeply considerate of the complexities and feedback-loops of the human body, over a doctor who is simply not smart enough to do so and, thus, can't. He can learn texts all he wants, but the memorization of text does not require deeper understanding.

But a smart person who hasn’t read all the texts won’t be a good doctor, either.

Chess players spend enormous amounts of time studying openings for a reason.

> Multiple small models, specifically trained for high reasoning/cognitive capabilities, given access to relevant texts

So, even assuming that one can train a model on reasoning/cognitive abilities, how does one pick the relevant texts for a desired outcome?

Re: Consistency diffusion language models: Up to 14x faster, no quality loss

#58

Post by turnsout »

adgjlsfhk1 wrote: post #54
irthomasthomas wrote: post #43

Earlier quoted context omitted.

You're hitting on something really important that barely gets discussed. For instance, notice how opus 4.5's speed essentially doubled, bringing it right in line with the speed of sonnet 4.5? (sonnet 4.6 got a speed bump too, though closer to 25%). It was the very first thing I noticed: it looks suspiciously like they just rebranded sonnet as opus and raised the price. I don't know why more people aren't talking abou…

opus 4.6 was going to be sonet 5 up until week of release. The price bump is even bigger than you realize because they don't let you run opus 4.6 at full speed unless you pay them an extra 10x for the new "fast mode"

If that's true, it would be surprising; the current Sonnet 4.6 is not in the same league as either Opus 4.5 or 4.6, either anecdotally or on benchmarks.

Re: Consistency diffusion language models: Up to 14x faster, no quality loss

#59

Post by turnsout »

LASR wrote: post #14
nl wrote: post #12

Releasing this on the same day as Taalas's 16,000 token-per-second acceleration for the roughly comparable Llama 8B model must hurt! I wonder how far down they can scale a diffusion LM? I've been playing with in-browser models, and the speed is painful. https://taalas.com/products/

Just tried this. Holy fuck. I'd take an army of high-school graduate LLMs to build my agentic applications over a couple of genius LLMs any day. This is a whole new paradigm of AI.

Man, I'm in the exact opposite camp. 1 smart model beats 1000 chaos monkeys any day of the week.
Post reply on HN