Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek V4 Pro 0813

openrouter.ai

Post reply on HN
Print view View original

281–290 of 493 posts

Re: DeepSeek V4 Pro 0813

#281

Post by p1necone »

simonw wrote: post #153
wolttam wrote: post #143

Earlier quoted context omitted.

I think I saw a better overall composition out of Flash 0731 Effort on this one?

Default effort for OpenRouter. I'll try a grid of efforts... Wow, the low, medium, and high pelicans came out in surprisingly different styles: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...

It's interesting that all three of those used roughly the same amount of tokens, and almost entirely output. Feels like the thinking level lever didn't alter cost at all for this specific task, even though it did change the output.

Re: DeepSeek V4 Pro 0813

#282

Post by sejje »

gnunez wrote: post #262
bigmadshoe wrote: post #198

Earlier quoted context omitted.

Why are people giving these n=1 comparisons like they mean anything? The worst offender is that pelican guy. These are non-deterministic systems and a single trial should not update your priors much at all. Of course it's significant that your response had a bug and took four times longer, but if you're only going to try once, this isn't real science, it's just vibes.

I don’t understand why people are calling these transformers models non-deterministic? Are you referring to the temperature parameter? I haven’t played with transformer internals in a while but my understanding is that if the temperature is fixed at a value where the top logit is always picked, then because they weights are fixed, the exact same input should produce the exact same output. Am I missing something?

I think people are wrapping that across the English language. In English, these two tasks are exactly the same:

"Would you hand me that item?"

"Please hand me that item"

But when posed to the LLM, they generate different outputs. One character difference in the prompt might be a whole different output. People who aren't programmers mostly don't know that there's any difference. They asked for the same thing, it knows what they want in both cases...but different results.

Re: DeepSeek V4 Pro 0813

#283

Post by fimi »

gnunez wrote: post #262
bigmadshoe wrote: post #198

Earlier quoted context omitted.

Why are people giving these n=1 comparisons like they mean anything? The worst offender is that pelican guy. These are non-deterministic systems and a single trial should not update your priors much at all. Of course it's significant that your response had a bug and took four times longer, but if you're only going to try once, this isn't real science, it's just vibes.

I don’t understand why people are calling these transformers models non-deterministic? Are you referring to the temperature parameter? I haven’t played with transformer internals in a while but my understanding is that if the temperature is fixed at a value where the top logit is always picked, then because they weights are fixed, the exact same input should produce the exact same output. Am I missing something?

https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

Re: DeepSeek V4 Pro 0813

#284

Post by neosat »

gnunez wrote: post #262
bigmadshoe wrote: post #198

Earlier quoted context omitted.

Why are people giving these n=1 comparisons like they mean anything? The worst offender is that pelican guy. These are non-deterministic systems and a single trial should not update your priors much at all. Of course it's significant that your response had a bug and took four times longer, but if you're only going to try once, this isn't real science, it's just vibes.

I don’t understand why people are calling these transformers models non-deterministic? Are you referring to the temperature parameter? I haven’t played with transformer internals in a while but my understanding is that if the temperature is fixed at a value where the top logit is always picked, then because they weights are fixed, the exact same input should produce the exact same output. Am I missing something?

As the above two comments mentioned this is not true in practice due to batch effects (you can read about some interesting work published by Thinking Machines on this), as well as calculation drift that happens across computations esp. now with inference optimization becoming common.

Re: DeepSeek V4 Pro 0813

#285

Post by roflmaostc »

khimaros wrote: post #280
simonw wrote: post #136

Nice bicycle chain, the little basket with a fish didn't show up in the right place: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...

these links never work for me. always "Error: Enter a valid URL" when opening in Firefox. maybe a URL escape issue with Glider?

my firefox works

Re: DeepSeek V4 Pro 0813

#286

Post by gnunez »

gpm wrote: post #265
gnunez wrote: post #262

Earlier quoted context omitted.

I don’t understand why people are calling these transformers models non-deterministic? Are you referring to the temperature parameter? I haven’t played with transformer internals in a while but my understanding is that if the temperature is fixed at a value where the top logit is always picked, then because they weights are fixed, the exact same input should produce the exact same output. Am I missing something?

Well, yes and no. By non-deterministic I think people really mean "chaotic" in the chaos theory sense. Small perturbations in the input lead to wild and unpredictable changes in the output. Even with temperature parameters a fixed PRNG seed could mean an LLM was just chaotic and not technically non-deterministic. But more literally while LLMs are in theory deterministic (though perhaps not inference providers impleme…

Ok. I see. I guess people are not referring to the raw models themselves when they say non-deterministic, but are also including the harness used in conjunction with the model. Then, in that case, for the exact same input you could get a non-deterministic output. But the model itself and all the mathematical machinery around the model is still very much deterministic.

I guess if we really needed to, we could construct a deterministic agent harness. But in most use cases we probably want some chaotic behavior to increase our chances of stumbling on the desired results.

Thank you for the clarification

Re: DeepSeek V4 Pro 0813

#287

Post by andy_ppp »

jug wrote: post #249
delduca wrote: post #234

Earlier quoted context omitted.

You don’t need the best model in 99% of cases…

This is true and is only becoming more important the more they improve. I am already moving to checking so they're at least somewhat following the status quo and otherwise prioritizing price and platform. I think this will be an emerging way of viewing AI in 2027 and the winner will probably be open models and China.

I think this likely plateaus and we all just get the smartest intelligence humans need running locally…

Re: DeepSeek V4 Pro 0813

#288

Post by gnunez »

nl wrote: post #272
gnunez wrote: post #262

Earlier quoted context omitted.

I don’t understand why people are calling these transformers models non-deterministic? Are you referring to the temperature parameter? I haven’t played with transformer internals in a while but my understanding is that if the temperature is fixed at a value where the top logit is always picked, then because they weights are fixed, the exact same input should produce the exact same output. Am I missing something?

> if the temperature is fixed at a value where the top logit is always picked, then because they weights are fixed, the exact same input should produce the exact same output. Am I missing something? Yes. Your input is part of a batch, and you don't know where in the batch it is. By default batches are not invariant and VLLM only supports invariance at all on some Huwaei Ascend hardware. See https://docs.vllm.ai/proje…

I totally missed the memo on batching. That changes everything. Thank you for the info.

Re: DeepSeek V4 Pro 0813

#289

Post by zxilly »

Palmik wrote: post #173

Why does this link to OpenRouter, which has no useful information on its own? Linking to the official API or the benchmarks would make more sense: - https://api-docs.deepseek.com/ - https://x.com/ChrisGPT/status/2087572834650407024/photo/1 (officially posted on WeChat, this is just one of many reposts)

There's no new page for this model. Hackernews didn't allow the same link be posted twice.

Re: DeepSeek V4 Pro 0813

#290

Post by Lalabadie »

p1necone wrote: post #279
monster_truck wrote: post #267

Have been letting it spin pretty hard (~$12.50 for 2B, 50% cache hits) on my traffic simulator/distributed physics engine all day, it's found some pretty significant gains without introducing any new problems. I'm happy

50% cache hit is really low - in a standard agentic loop you should expect like 99%+ cache hit percentage (which should also lower that $12.50 to like a couple of $ for the same amount of tokens). If you're using a customised harness you should make sure you don't have something that's e.g. changing your system prompt on some requests or rewriting history - it can be tempting to do stuff like strip old thinking token…

In my experience, that's the OpenRouter tax. Even a session that does everything right to remain sticky ends up getting moved between providers on a few requests, which bills you the full context as input every time the switch happens.

I assume it's done as load balancing/latency mitigation, but it's put me off of OpenRouter for my use cases (limited use, limited need for changing models).

Post reply on HN