Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mercury: Commercial-scale diffusion language model

inceptionlabs.ai

Post reply on HN
Print view View original

151–160 of 189 posts

Re: Mercury: Commercial-scale diffusion language model

#151

Post by crazygringo »

jonplackett wrote: post #116
crazygringo wrote: post #16

Earlier quoted context omitted.

For me, ChatGPT (the free version, GPT-4o mini I believe?) gets it right, choosing option 2 because the coffee will cool faster due to the larger temperature difference. Unless there's a gotcha somewhere in your prompt that I'm missing, like what if the temperature of the room is hotter than the coffee, or so cold that the coffee becomes colder than the milk, or something? I would be suprised if any models get it wro…

This is what I got from full-fat 4o. Maybe thinking less helps! ChatGPT: Option 1 — Add the cold milk immediately — will result in a lower final temperature after 2 minutes. Why: • Heat loss depends on the temperature difference between the coffee and the environment (usually room temperature). • If you add the milk early, the overall temperature of the coffee-milk mixture is reduced immediately. This lowers the aver…

Oof. I wonder what makes it so bad?

In my experience LLM's tend to be pretty good at basic logic as long as they understand the domain well enough.

I mean, it even gets it right at first -- "This lowers the average temperature over the 2 minutes, so less heat is lost to the air." -- but then it seems to get conceptually confused about heat loss vs cooling, which is surprising.

Re: Mercury: Commercial-scale diffusion language model

#152

Post by lostmsu »

badmonster wrote: post #56

1000+ tokens/sec on H100s, a 5–10x speedup over typical autoregressive models — and without needing exotic hardware like Groq or Cerebras - impressive

Would batch inference increase throughput further? Or does it already peak the FLOPS?

Re: Mercury: Commercial-scale diffusion language model

#153

Post by [deleted] »

pama wrote: post #130
kmacdough wrote: post #126

Earlier quoted context omitted.

What are the barriers to mixed architecture models? Models which could seamlessly pass from autoregressive to diffusion, etc. Humans can integrate multiple sensory processing centers and multiple modes of thought all at once. It's baked into our training process (life).

The human processing is still autoregressive, but using multiple parallel synchronized streams. There is no problem with such an approach and my best guess is that in the next year we will see many teams training models using such tricks for generating reasoning traces in parallel. The main concern is taking a single probabilistic stream (eg a book) and comparing autoregressive modelling of it with a diffusive modell…

[deleted]

Re: Mercury: Commercial-scale diffusion language model

#154

Post by vlovich123 »

kmacdough wrote: post #129
vlovich123 wrote: post #101

I just tried giving it a coding snippet that has a bug. ChatGPT & Claude found the bug instantly. Mercury fails to find it even after several reprompts (it's hallucinating). On the upside it is significantly faster. That's promising since the edge for ChatGPT and Claude are in the prolonged time and energy they've spent building training infrastructure, tooling, datasets, etc to pump out models with high task perform…

Keep in mind this release was never intended to prove superiority. Rather, it shows an alternative structure with some promising performance characteristics. More work needs to be done to show real application, but this very valuable learning. That's part of the reason to compare against older, smaller models since they're at a more comparable stage of development.

I agree. As I was trying to imply, I think if you integrated this structure into OpenAI’s or Claude’s stack, you’d get a vastly cheaper model that’s significantly faster with similar task performance (modulo the structural task performance parts that are hard to port to this new architecture). The point about quality was also intended to temper some of the excitement about the scores published on the page.

Re: Mercury: Commercial-scale diffusion language model

#156

Post by kazinator »

jonplackett wrote: post #114
kazinator wrote: post #68

Earlier quoted context omitted.

That's totally cribbed from some discussion hat occurred in its training.

If it was just ‘in the training data’ they’d all get it right. But they don’t.

I don't think that can be postulated as a law, because they are a kind of lossy compression. Different lossy compressions will lose different details.

Re: Mercury: Commercial-scale diffusion language model

#157

Post by geysersam »

kadushka wrote: post #150
jacob019 wrote: post #146

Earlier quoted context omitted.

Because it's easy and cheap. Like how many products use a Raspberry Pi or ESP32 when an ATtiny would do.

How in the world is this easy and cheap? Are you planning to run this LLM inside the vending machine? Or are you planning to send those prompts to a remote LLM somewhere?

The premise here is that the model runs fast and cheap. With the current state of the technology running a vending machine using an LLM is of course absurd. The point is that accuracy is not the only dimension that brings qualitative change to the kind of applications that LLMs are useful for.

Re: Mercury: Commercial-scale diffusion language model

#158

Post by otabdeveloper4 »

geysersam wrote: post #143
otabdeveloper4 wrote: post #108

Earlier quoted context omitted.

> and then ask it "why did you just now give that customer mountain dew when they ordered sprite?" Worse than useless for debugging. An LLM can't think and doesn't have capabilities for self-reflection. It will just generate a plausible stream of tokens in reply that may or may not correspond to the real reason why.

Of course a llm can't think. But that doesn't mean it can't answer simple questions about the output that was produced. Just try it out with chatgpt when you have time. Even if it's not perfectly accurate it's still useful for debugging. Just think about it as a human employee. Can they always say why they did what they did? Often, but not always. Sometimes you will have to work to figure out the misunderstanding.

> it's still useful for debugging

How so? What the LLM says is whatever is more likely given the context. It has no relation to the underlying reality whatsoever.

Re: Mercury: Commercial-scale diffusion language model

#159

Post by otabdeveloper4 »

IshKebab wrote: post #105
otabdeveloper4 wrote: post #103

Earlier quoted context omitted.

LLM's can't think, so "smarter" is not possible.

They can by the normal English definitions of "think" and "smart". You're just redefining those words to exclude AI because you feel threatened by it. It's tedious.

Incorrect. LLM's have no self-reflection capability. That's a key prerequisite for "thinking". ("I think, therefore I am.")

They are simple calculators that answer with whatever tokens are most likely given the context. If you want reasonable or correct answers (rather than the most likely) then you're out of luck.

Re: Mercury: Commercial-scale diffusion language model

#160

Post by mhh__ »

suddenlybananas wrote: post #138
mhh__ wrote: post #134

Earlier quoted context omitted.

We get way more info than llms do, just not solely from text

You have not read every accessible piece of text in existence.

There is more to life than just text e.g. this is part of lecun argument against LLMs
Post reply on HN