Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mercury: Commercial-scale diffusion language model

inceptionlabs.ai

Post reply on HN
Print view View original

71–80 of 189 posts

Re: Mercury: Commercial-scale diffusion language model

#71

Post by mhh__ »

kazinator wrote: post #68
jefftk wrote: post #54

Earlier quoted context omitted.

Claude 3.7 gets it exactly right: To determine which option cools coffee the most, I'll analyze the heat transfer physics involved. The key insight is that the rate of heat loss depends on the temperature difference between the coffee and the surrounding air. When the coffee is hotter, it loses heat faster. Option 1 (add milk first, then wait): - Adding cold milk immediately lowers the coffee temperature right away -…

That's totally cribbed from some discussion hat occurred in its training.

So is my knowledge of newtons law of cooling

Re: Mercury: Commercial-scale diffusion language model

#72

Post by selcuka »

adammarples wrote: post #27
jonplackett wrote: post #5

Ok. My go to puzzle is this: You have 2 minutes to cool down a cup of coffee to the lowest temp you can You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add the cold milk. Which one cools the coffee to the lowest temperature and why? And Mercury gets this right - while as of right now ChatGPT 4o get it wrong. So that’s pretty impressive.

If you let it sit for 2 minutes your time is up and you don't have time to add the cold milk

By this logic you can't let it sit for 2 mins after you add the cold milk either, so both options are invalid.

In math/science questions some things are assumed to be (practically impossibly) instant.

Re: Mercury: Commercial-scale diffusion language model

#73

Post by adrianmonk »

firejake308 wrote: post #51
drusepth wrote: post #32

Earlier quoted context omitted.

Is there a trick question in here, where the cold milk is trending closer to room temperature over the two minutes waited?

The rate of change of temperature is proportional to the difference in temperature. Adding the cold milk later let's you juice a larger temperature differential out of the 2-minute waiting phase. So you should add the milk later.

> the cold milk later

I think what the other person is asking about is: can you be sure that the milk is (as) cold later?

There's a lot of discussion about what happens to the temperature of the coffee during the 2 minutes. What happens to the temperature of the milk during that same time?

Where is the milk stored? Do you grab it out of the refrigerator the moment you add it to the coffee? Or the cold milk sitting out on the countertop getting warmer? If so, how rapidly?

Re: Mercury: Commercial-scale diffusion language model

#75

Post by Nevermark »

kazinator wrote: post #68
jefftk wrote: post #54

Earlier quoted context omitted.

Claude 3.7 gets it exactly right: To determine which option cools coffee the most, I'll analyze the heat transfer physics involved. The key insight is that the rate of heat loss depends on the temperature difference between the coffee and the surrounding air. When the coffee is hotter, it loses heat faster. Option 1 (add milk first, then wait): - Adding cold milk immediately lowers the coffee temperature right away -…

That's totally cribbed from some discussion hat occurred in its training.

As apposed to humans who all derive the physics of heat transfer independently when given a question like this?

Not picking on you - this brings up something we could all get better at:

There should be a "First Rule of Critiquing Models": Define a baseline system to compare performance against. When in doubt, or for general critiques of models, compare to real world random human performance.

Without a real practical baseline to compare with, its to easy to fall into subjective or unrealistic judgements.

"Second Rule": Avoid selectively biasing judgements by down selecting performance dimensions. For instance, don't ignore difference in response times, grammatical coherence, clarity of communication, and other qualitative and quantitative differences. Lack of comprehensive performance dimension coverage is like comparing runtimes of runners, without taking into account differences in terrain, length of race, altitude, temperature, etc.

It is very easy to critique. It is harder to critique in a way that sheds light.

Re: Mercury: Commercial-scale diffusion language model

#76

Post by kittikitti »

This is genius! There are tradeoffs between diffusion and neural network models in image generation so why not use diffusion models in text generation? Excited to see where this ends up and I wouldn't be surprised if we saw some of these types of models appear in the future updates to popular families like Llama or Qwen.

Re: Mercury: Commercial-scale diffusion language model

#77

Post by kittikitti »

inerte wrote: post #7

Not sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I hav…

I don't use the best available models for prototyping because it can be expensive or more time consuming. This innovation makes prototyping faster and practicing prompts on slightly lower accuracy models can provide more realistic expectations.

Re: Mercury: Commercial-scale diffusion language model

#78

Post by twotwotwo »

It's kind of weird to think that in a coding assistant, an LLM is regularly asked to produce a valid block of code top to bottom, or repeat a section of code with changes, when that's not what we do. (There are other intuitively odd things about this, like the amount of compute spent generating 'easy' tokens, e.g. repeating unchanged code.) Some of that might be that models are just weird and intuition doesn't apply. But maybe the way we do it--jumping around, correcting as we go, etc.--is legitimately an efficient use of effort, and a model could do its job better, with less effort, or both if it too used some approach other than generating the whole sequence start-to-finish.

There's already stuff in the wild moving that direction without completely rethinking how models work. Cursor and now other tools seem to have models for 'next edit' not just 'next word typed'. Agents can edit a thing and then edit again (in response to lints or whatever else); approaches based on tools and prompting like that can be iterated on without the level of resources needed to train a model. You could also imagine post-training a model specifically to be good at producing edit sequences, so it can actually 'hit backspace' or replace part of what it's written if it becomes clear it wasn't right, or if two parts of the output 'disagree' and need to be reconciled.

From a quick search it looks like https://arxiv.org/abs/2306.05426 in 2023 discussed backtracking LLMs and https://arxiv.org/html/2410.02749v3 / https://github.com/upiterbarg/lintseq trained models on synthetic edit sequences. There is probably more out there with some digging. (Not the same topic, but the search also turned up https://arxiv.org/html/2504.20196 from this Monday(!) about automatic prompt improvement for an internal code-editing tool at Google.)

Re: Mercury: Commercial-scale diffusion language model

#79

Post by kazinator »

mhh__ wrote: post #71
kazinator wrote: post #68

Earlier quoted context omitted.

That's totally cribbed from some discussion hat occurred in its training.

So is my knowledge of newtons law of cooling

If an LLM has only that knowledge and nothing else (pieces of text saying that heat transfer is proportional to some function of the temp difference) such that is not trained on any texts that give problems and solutions in this area, it will not work this out, since it has nothing to generate tokens from.

Also, your knowledge doesn't come from anywhere near having scanned terabytes of text, which would take you multiple lifetimes of full time work.

Re: Mercury: Commercial-scale diffusion language model

#80

Post by fizx »

XenophileJKO wrote: post #29
inerte wrote: post #7

Not sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I hav…

So my personal belief is that diffusion models will enable higher degrees of accuracy. This is because unlike an auto-regressive model it can adjust a whole block of tokens when it encounters some kind of disjunction. Think of the old example where an auto regressive model would output: "There are 2 possibilities.." before it really enumerated them. Often the model has trouble overcoming the bias and will hallucinate…

Once that auto-regressive model goes deep enough (or uses "reasoning"), it actually has modeled what possibilities exist by the time it's said "There are 2 possibilities.."

We're long past that point of model complexity.

Post reply on HN