Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mercury: Commercial-scale diffusion language model

inceptionlabs.ai

Post reply on HN
Print view View original

161–170 of 189 posts

Re: Mercury: Commercial-scale diffusion language model

#161

Post by kadushka »

geysersam wrote: post #157
kadushka wrote: post #150

Earlier quoted context omitted.

How in the world is this easy and cheap? Are you planning to run this LLM inside the vending machine? Or are you planning to send those prompts to a remote LLM somewhere?

The premise here is that the model runs fast and cheap. With the current state of the technology running a vending machine using an LLM is of course absurd. The point is that accuracy is not the only dimension that brings qualitative change to the kind of applications that LLMs are useful for.

Running a vending machine using an LLM is absurd not because we can't run LLMs fast or cheap enough - it's because LLMs are not reliable, and we don't know yet how to make them more reliable. Our best LLM - o3 - doubled the previous model (o1) hallucination rate. OpenAI says it hallucinated a wrong answer 33% of the time in benchmarks. Do you want a vending machine that screws up 33% of the time?

Today, the accuracy of LLMs is by far a bigger concern (and a harder problem to solve) than its speed. If someone releases a model which is 10x slower than o3, but is 20% better in terms of accuracy, reliability, or some other metric of its output quality, I'd switch to it in a heartbeat (and I'd be ready to pay more for it). I can't wait until o3-pro is released.

Re: Mercury: Commercial-scale diffusion language model

#162

Post by kazinator »

mhh__ wrote: post #160
suddenlybananas wrote: post #138

Earlier quoted context omitted.

You have not read every accessible piece of text in existence.

There is more to life than just text e.g. this is part of lecun argument against LLMs

There is more to text than just predicting tokens based on a vast volume of text.

There isn't an argument "against LLMs" as such; the argumentation is more oriented against the hype and incessant promotion of AI.

Re: Mercury: Commercial-scale diffusion language model

#163

Post by geysersam »

otabdeveloper4 wrote: post #158
geysersam wrote: post #143

Earlier quoted context omitted.

Of course a llm can't think. But that doesn't mean it can't answer simple questions about the output that was produced. Just try it out with chatgpt when you have time. Even if it's not perfectly accurate it's still useful for debugging. Just think about it as a human employee. Can they always say why they did what they did? Often, but not always. Sometimes you will have to work to figure out the misunderstanding.

> it's still useful for debugging How so? What the LLM says is whatever is more likely given the context. It has no relation to the underlying reality whatsoever.

Not sure what you mean by "relation to the underlying reality". The explanation is likely to be correlated with the underlying reason for the answer.

For example, here is a simple query:

> I put my bike in the bike stand, I removed my bike from the bike stand and biked to the beach, then I biked home, where is my bike. Answer only with the answer and no explanation

> Chatgpt: At home

> Why did you answer that?

> I answered that your bike is at home because the last action you described was biking home, which implies you took the bike with you and ended your journey there. Therefore, the bike would logically be at home now.

Do you doubt that the answer would change if I changed the query to make the final destination be "the park" instead of "home"? If you don't doubt that, what do you mean that the answer doesn't correspond to the underlying reality? The reality is the answer depends on the final destination mentioned, and that's also the explanation given by the LLM, clearly the reality and the answers are related.

Re: Mercury: Commercial-scale diffusion language model

#164

Post by albertzeyer »

freeqaz wrote: post #53

Anybody able to get the "View Technical Report" button at the bottom to do anything? I was curious to glean more details but it doesn't work on either of my devices. I'm curious what level of detail they're comfortable publishing around this, or are they going full secret mode?

It links to this file: https://drive.google.com/file/d/1j1ofmm8iBaVreGC5TSF1oLsrOqB...

But all but the first page seems to be missing in this PDF? There is just an abstract and (partial) outline.

Re: Mercury: Commercial-scale diffusion language model

#165

Post by cratermoon »

jonplackett wrote: post #117
cratermoon wrote: post #33

Earlier quoted context omitted.

> My go to puzzle is this: > Mercury gets this right - while as of right now ChatGPT 4o get it wrong. This is so common a puzzle it's discussed all over the internet. It's in the data used to build the models. What's so impressive about a machine that can spit out something easily found with a quick web search?

Just that what I thought would be better models don’t do it right. I was expecting this model to be no-where near chatGPT Although someone above is saying 4o-mini got it right so maybe it’s meaningless. Or maybe thinking less helps…

There is sufficient stochasticity in LLMs to invalidate most comparisons at this level. Minor changes in the prompt text, even from run to run in the same model, will produce different results (depending on temperature and other paramters), much less different models.

Try re-running your test on the same model multiple times with the identical prompt, or varying the prompt. Depending on how much context the service you choose is keeping for you across a conversation, the behavior can change. Something as simple as prompting an incorrect response with a request to try again because the result was wrong can give different results.

Statistically, the model will eventually hit on the right combination of vectors and generate the right words from the training set, and as I noted before, this problem has a very high probability of being in the training data used to build all the models easily available.

Re: Mercury: Commercial-scale diffusion language model

#166

Post by thomastjeffery »

inerte wrote: post #7

Not sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I hav…

Accuracy is a myth.

These models do not reason. They do not calculate. They perform no objectivity whatsoever.

Instead, these models show us what is most statistically familiar. The result is usually objectively sound, or at least close enough that we can rewrite it as something that is.

Re: Mercury: Commercial-scale diffusion language model

#167

Post by K0balt »

kadushka wrote: post #150
jacob019 wrote: post #146

Earlier quoted context omitted.

Because it's easy and cheap. Like how many products use a Raspberry Pi or ESP32 when an ATtiny would do.

How in the world is this easy and cheap? Are you planning to run this LLM inside the vending machine? Or are you planning to send those prompts to a remote LLM somewhere?

You could run a 3B model on 200 dollars worth of hardware and it would do just fine, 100 percent of the time, most of the time. I could definitely see someone talking it out of a free coke now and then though.

With vending machines costing 2-5k, it’s not out of the question, but it’s hard to imagine the business case for it. Maybe the tantalizing possibility of getting a free soda would attract traffic and result in additional sales from frustrated grifters? Idk.

Re: Mercury: Commercial-scale diffusion language model

#168

Post by accrual »

Nevermark wrote: post #75
kazinator wrote: post #68

Earlier quoted context omitted.

That's totally cribbed from some discussion hat occurred in its training.

As apposed to humans who all derive the physics of heat transfer independently when given a question like this? Not picking on you - this brings up something we could all get better at: There should be a "First Rule of Critiquing Models": Define a baseline system to compare performance against. When in doubt, or for general critiques of models, compare to real world random human performance. Without a real practical…

> It is very easy to critique. It is harder to critique in a way that sheds light.

Well said. This is the sort of ethos I admire and aspire to on HN.

Re: Mercury: Commercial-scale diffusion language model

#169

Post by cchance »

pama wrote: post #130
kmacdough wrote: post #126

Earlier quoted context omitted.

What are the barriers to mixed architecture models? Models which could seamlessly pass from autoregressive to diffusion, etc. Humans can integrate multiple sensory processing centers and multiple modes of thought all at once. It's baked into our training process (life).

The human processing is still autoregressive, but using multiple parallel synchronized streams. There is no problem with such an approach and my best guess is that in the next year we will see many teams training models using such tricks for generating reasoning traces in parallel. The main concern is taking a single probabilistic stream (eg a book) and comparing autoregressive modelling of it with a diffusive modell…

Maybe diffusion for "thoughts" and autoregressive for output :S

Re: Mercury: Commercial-scale diffusion language model

#170

Post by IshKebab »

otabdeveloper4 wrote: post #159
IshKebab wrote: post #105

Earlier quoted context omitted.

They can by the normal English definitions of "think" and "smart". You're just redefining those words to exclude AI because you feel threatened by it. It's tedious.

Incorrect. LLM's have no self-reflection capability. That's a key prerequisite for "thinking". ("I think, therefore I am.") They are simple calculators that answer with whatever tokens are most likely given the context. If you want reasonable or correct answers (rather than the most likely) then you're out of luck.

It is not a key prerequisite for "thinking". It's "I think therefore I am" not "I am self-aware therefore I think".

In the 90s if your cursor turned into an hourglass and someone said "it's thinking" would you have pedantically said "NO! It is merely calculating!"

Maybe you would... but normal people with normal English would not.

Post reply on HN