I just tried giving it a coding snippet that has a bug. ChatGPT & Claude found the bug instantly. Mercury fails to find it even after several reprompts (it's hallucinating). On the upside it is significantly faster. That's promising since the edge for ChatGPT and Claude are in the prolonged time and energy they've spent building training infrastructure, tooling, datasets, etc to pump out models with high task performance.
Mercury: Commercial-scale diffusion language model
101–110 of 189 posts
Re: Mercury: Commercial-scale diffusion language model
#102Not sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I hav…
> I now realize I should have done things incrementally even when I have a pretty good idea of the final picture.
Or just save yourself the time and money and code it yourself like it's 2020.
(Unless it's your employer paying for this waste, in which case go for it, I guess.)
Re: Mercury: Commercial-scale diffusion language model
#103Not sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I hav…
AI field desperately needs smarter models - not faster models.
LLM's can't think, so "smarter" is not possible.
Re: Mercury: Commercial-scale diffusion language model
#104It's nice to see a team doing something different. The cost[1] is US$1.00 per million output tokens and US$0.25 per million input tokens. By comparison, Gemini 2.5 Flash Preview charges US$0.15 per million tokens for text input and $0.60 (non-thinking) output[2]. Hmmm... at those prices they need to focus on markets where speed is especially important, eg high-frequency trading, transcription/translation services and…
Not sure how HFTs are relevant here
HFT is limited by time on how much processing it can do. In theory, a super-fast dLLM would enable to incorporate information sources in their decision-making that were previously too high-level. E.g., imagine using wire reports to predict an arbitrage opportunity that doesn't even exist yet (I dunno, not an HFT guy).
In practice, iiuc, HFT still happens within 10s of milliseconds, and I doubt even current dLLM is THAT fast.
Re: Mercury: Commercial-scale diffusion language model
#105Earlier quoted context omitted.
AI field desperately needs smarter models - not faster models.
LLM's can't think, so "smarter" is not possible.
They can by the normal English definitions of "think" and "smart". You're just redefining those words to exclude AI because you feel threatened by it. It's tedious.
Re: Mercury: Commercial-scale diffusion language model
#106Looks interesting, and my intuition is that code is a good application of diffusion LLMs, especially if they get support for "constrained generation", as there's already plenty of tooling around code (linters and so on).
Something I don't see explored in their presentation is the ability of the model to restore from errors / correct itself. SotA LLMs shine at this, a few back and forth w/ sonnet / gemini pro / etc really solves most problems nowadays.
Re: Mercury: Commercial-scale diffusion language model
#107Ok. My go to puzzle is this: You have 2 minutes to cool down a cup of coffee to the lowest temp you can You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add the cold milk. Which one cools the coffee to the lowest temperature and why? And Mercury gets this right - while as of right now ChatGPT 4o get it wrong. So that’s pretty impressive.
Depends on the shape of the cup! You can contrive a cup shaped like an exponentially flaring horn, where adding the milk increases the volume a little, which massively increases the surface area, and so leads to faster cooling. Or you can have a cup with a converging top, like a brandy glass, where adding the milk reduces the surface area, and makes cooling even slower.
Clever, but it's implicitly a normal cylindrical cup.
Re: Mercury: Commercial-scale diffusion language model
#108Earlier quoted context omitted.
I think natural language to code is the right abstraction. Easy enough barrier to entry but still debuggable. Debugging why an LLM randomly gives you Mountain Dew instead of Sprite if you have a southern accent sounds like a nightmare.
I'm not sure it would be that hard to debug. Make sure you can reproduce the llm state (by storing the random seed for the session, or something like that) and then ask it "why did you just now give that customer mountain dew when they ordered sprite?"
> and then ask it "why did you just now give that customer mountain dew when they ordered sprite?"
Worse than useless for debugging.
An LLM can't think and doesn't have capabilities for self-reflection.
It will just generate a plausible stream of tokens in reply that may or may not correspond to the real reason why.
Re: Mercury: Commercial-scale diffusion language model
#109Not sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I hav…
AI field desperately needs smarter models - not faster models.
If you have much more speed in the available time, for an activity like coding, you could use that for iteration, writing more tests and satisfying them, especially if you can pair that with a concurrent test runner to provide feedback. I'm not sure the end result would be lower scoring/smartness than an LLM could achieve in the same duration.
Re: Mercury: Commercial-scale diffusion language model
#110I'd hope that with diffusion, it would be able to go back and forth between parts of the output to adjust issues with part of the output which it had previously generated. This would not be possible with a purely sequential model.
However,
> Prompt: Write a sentence with ten words which has exactly as many r’s in the first five words as in the last five
>
> Response: Rapidly running, rats rush, racing, racing.