Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mercury: Commercial-scale diffusion language model

inceptionlabs.ai

Post reply on HN
Print view View original

11–20 of 189 posts

Re: Mercury: Commercial-scale diffusion language model

#11

Post by orbital-decay »

parsimo2010 wrote: post #8

This sounds like a neat idea but it seems like bad timing. OpenAI just released token-based that beats the best diffusion image generation. If diffusion isn't even the best at generating images, I don't know if I'm going to spend a lot of time evaluating it for text. Speed is great but it doesn't seem like other text-based model trends are going to work out of the box, like reasoning. So you have to get dLLMs up to t…

Does it beat them because it's a transformer, or because it's a much larger end-to-end model with higher quality multimodal training?

Re: Mercury: Commercial-scale diffusion language model

#12

Post by jonplackett »

parsimo2010 wrote: post #8

This sounds like a neat idea but it seems like bad timing. OpenAI just released token-based that beats the best diffusion image generation. If diffusion isn't even the best at generating images, I don't know if I'm going to spend a lot of time evaluating it for text. Speed is great but it doesn't seem like other text-based model trends are going to work out of the box, like reasoning. So you have to get dLLMs up to t…

The reason image-1 is so good is because it’s the same model doing the talking and the image making.

I wonder if the same would be true for a multi-modal diffusion model that can now also speak?

Re: Mercury: Commercial-scale diffusion language model

#13

Post by pants2 »

This is awesome for the future of autocomplete. Current models aren't fast enough to give useful suggestions at the speed that I type - but this certainly is.

That said, token-based models are currently fast enough for most real-time chat applications, so I wonder what other use-cases there will be where speed is greatly prioritized over smarts. Perhaps trading on Trump tweets?

Re: Mercury: Commercial-scale diffusion language model

#14

Post by marcyb5st »

whall6 wrote: post #9
marcyb5st wrote: post #6

Super happy to see something like this getting traction. As someone that is trying to reduce my carbon footprint sometimes I feel bad about asking any model to do something trivial. With something like that perhaps the guilt will lessen

If you live in the U.S., marginal electricity demand during the day is almost invariably met with solar or wind (solar typically runs at a huge surplus on sunny days). Go forth and AI in peace, marcyb5st.

Thanks! That helps somewhat. However, it feels like that's just part of the story.

If I remember correctly hyperscalers put their green agendas in stasis now that LLMs are around and that makes me believe that there is a CO2 cost associated.

Still, any improvement is a good news and if diffusion models replace autoregressive models we can invest that surplus in energy in something else useful for the environment.

Re: Mercury: Commercial-scale diffusion language model

#15

Post by nullc »

byearthithatius wrote: post #4

Interesting approach. However, I never thought of auto regression being _the_ current issue with language modeling. If anything it seems the community was generally surprised just how far next "token" prediction took us. Remember back when we did char generating RNNs and were impressed they could make almost coherent sentences? Diffusion is an alternative but I am having a hard time understanding the whole "built in…

Consider the entropy of the distribution of token X in these examples:

"Four X"

and

"Four X and seven years ago".

In the first case X could be pretty much anything, but in the second case we both know the only likely completion.

So it seems like there would be a huge advantage in not having to run autogressively. But in practice it's less significant then you might imagine because the AR model can internally model the probability of X conditioned on the stuff it hasn't output yet, and in fact because without reinforcement the training causes it converge on the target probability of the whole output, the AR model must do some form of lookahead internally.

(That said RLHF seems to break this product of the probabilities property pretty badly, so maybe it will be the case that diffusion will suffer less intelligence loss ::shrugs::).

Re: Mercury: Commercial-scale diffusion language model

#16

Post by crazygringo »

jonplackett wrote: post #5

Ok. My go to puzzle is this: You have 2 minutes to cool down a cup of coffee to the lowest temp you can You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add the cold milk. Which one cools the coffee to the lowest temperature and why? And Mercury gets this right - while as of right now ChatGPT 4o get it wrong. So that’s pretty impressive.

For me, ChatGPT (the free version, GPT-4o mini I believe?) gets it right, choosing option 2 because the coffee will cool faster due to the larger temperature difference.

Unless there's a gotcha somewhere in your prompt that I'm missing, like what if the temperature of the room is hotter than the coffee, or so cold that the coffee becomes colder than the milk, or something?

I would be suprised if any models get it wrong, since I assume it shows up in training data a bunch?

Re: Mercury: Commercial-scale diffusion language model

#17

Post by maytc »

jonplackett wrote: post #5

Ok. My go to puzzle is this: You have 2 minutes to cool down a cup of coffee to the lowest temp you can You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add the cold milk. Which one cools the coffee to the lowest temperature and why? And Mercury gets this right - while as of right now ChatGPT 4o get it wrong. So that’s pretty impressive.

That example is probably in the training data?

The puzzle assumes that the room temperature is greater than the cold milk's temperature. When I added that the room temperature is, say, -10 °C, Mercury fails to see the difference.

Re: Mercury: Commercial-scale diffusion language model

#18

Post by kadushka »

inerte wrote: post #7

Not sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I hav…

AI field desperately needs smarter models - not faster models.

Re: Mercury: Commercial-scale diffusion language model

#19

Post by tyre »

inerte wrote: post #7

Not sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I hav…

Check out RooCode if you haven’t. There’s an orchestrator mode that can start with a big model to come up with a plan and break down, then spin out small tasks to smaller models for scoped implementation.

Re: Mercury: Commercial-scale diffusion language model

#20

Post by krackers »

jonplackett wrote: post #5

Ok. My go to puzzle is this: You have 2 minutes to cool down a cup of coffee to the lowest temp you can You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add the cold milk. Which one cools the coffee to the lowest temperature and why? And Mercury gets this right - while as of right now ChatGPT 4o get it wrong. So that’s pretty impressive.

Hmm a good nerd-snipe puzzle. I was never very good at physics, so hopefully someone can check my work... assuming upon mixing coffee is at Tc and milk at Tm, and simplifying to assume equivalent mass & specific temp we have (Tf - Tc) = -(Tf - Tm) => Tf = (Tc+Tm)/2 which is intuitive (upon mixing we get the average temperature).

On the assumption that the cold milk is always at a fixed temperature until it's mixed in, then the temperature of coffee at point of mixing is the main factor. Before and after, it follows newton's law of cooling. So we're comparing something like Tenv + [(Tc+Tm)/2 - Tenv]e^(-2) vs (Tenv + [Tc - Tenv]e^(-2) + Tm)/2. The latter is greater than the former only when Tm > Tenv (the milk isn't cold), or in other words it's better to let the coffee cool as much as possible before mixing assuming the milk is colder than the environment.

Another interesting twist is to consider the case where the milk isn't kept at a fixed temperature but is also subject to warming (it's taken out of the fridge). Then the former equation is unchanged but the latter becomes (Tenv + [Tc - Tenv]e^(-2) + Tenv + [Tm - Tenv]e^(-2))/2. But this is equivalent to the former equation, so in this case it doesn't matter when you mix it.

Not 100% confident in both analysis, but I wonder if there's a more intuitive way to see it. I also don't know if deviating from the assumption of equivalent mass & specific temp changes the analysis (it might lead to a small range where for the fixed case, situation 1 is better?) It's definitely not "intuitive" to me.

Post reply on HN