Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek V4 Pro 0813

openrouter.ai

Post reply on HN
Print view View original

211–220 of 493 posts

Re: DeepSeek V4 Pro 0813

#211

Post by jklmnopqrstuvw »

bigmadshoe wrote: post #198
jklmnopqrstuvw wrote: post #65

Tested both DS v4 pro 0813 and Grok 4.6 (all from openrouter) on Codex cli. Worked on a same new feature development on my project. Deepseek 4 pro: Worked for 12m 02s - cost $0.12 - has bug. Grok 4.6: Worked for 3m 18s - cost $ 1.41 - no bug.

Why are people giving these n=1 comparisons like they mean anything? The worst offender is that pelican guy. These are non-deterministic systems and a single trial should not update your priors much at all. Of course it's significant that your response had a bug and took four times longer, but if you're only going to try once, this isn't real science, it's just vibes.

Months ago I start making this kind of test for my own reference. At beginning I I test each model multiple times, and results always same(pass or fail). Later I test only once for new models, I trust the results.

Re: DeepSeek V4 Pro 0813

#212

Post by computerex »

segmondy wrote: post #178
computerex wrote: post #164

Earlier quoted context omitted.

They are causal autoregressive models, the output is sensitive even to the implementation nuances in inference. Even 1 token that's badly selected could throw off the entire answer.

you're thinking of one shot. if they are running an agentic loop then they don't need multiple passes. an agentic loop is multiple passes with tool calls and tools could fail and agent would correct from seeing the failure. a bad model will compound on error and fail, a good model will correct. 1 test is fine to gauge the quality of the model.

An agent doing a task even with multiple back to back calls like normal without an example is zero shot. An agent doing a task with 1 example is one shot. An agent doing a task with a few examples is few shot. I don't think you are correctly using these terms.

The multiple back to back LLM calls are done on accumulating context, so if there is a sampling error it could throw the entire session out of whack, because LLM's build on the previous context.

It's actually meaningless to argue, one could simply sample more than 1 times and let the numbers speak for themselves.

Re: DeepSeek V4 Pro 0813

#213

Post by greenchair »

hgoel wrote: post #105
bikemike026 wrote: post #89

Earlier quoted context omitted.

If you read Opus 5's output, it is beyond the comprehension of virtually all engineers and developers. That is what I mean by intelligence. Math, science, and engineering are all contained in one model. We may be experts in one field. The model is an expert in everything that humans know.

I don't think that's because of its "intelligence". It speaks obtuse techbro-ese: stringing together words that sound smart to obscure the simplicity of the thing it's describing. In many ways it's the opposite of intelligence. Opus 5 and Fable 5 in particular suffer from this issue at worse level than most models in the same class.

yep, it is so bad i had to create rules to cut down on the techbro language and domain slang.

Re: DeepSeek V4 Pro 0813

#215

Post by Eueudhsbsj32 »

akman wrote: post #190
Eueudhsbsj32 wrote: post #79

Earlier quoted context omitted.

Unless you're Chinese, why would you care if they see your data? As an American, I'd much rather have my data kept outside the country than here where companies and the government have a lot more leverage over me.

I do think this question comes up a lot-- I can understand why. For some well-explained reasons, check out https://darioamodei.com/essay/the-adolescence-of-technology and search for "CCP".

So is your concern more about reducing the risk of an authoritarian China "winning" the AI race? And less about reducing the risk of your data being used against you personally?

To me, the risks of an individual helping China to continue to develop their AI by being a customer is pretty marginal compared with the personal risks of my data being used against me.

Re: DeepSeek V4 Pro 0813

#216

Post by taosx »

RALaBarge wrote: post #203
taosx wrote: post #152

Earlier quoted context omitted.

I created a simulation for coding harnesses based on my own pi sessions. When taking into account all factors, DS-v4-Pro is cheaper than gpt-5.6-luna due to caching. Look at the bill segments difference for cache read cost and uncached cost between deepseek and the other models. At this point is cheaper to use ds-v4-pro than the luna models from openai. ignore the numbers except the classic and keep in mind that clas…

Hey this looks good! Maybe consider adding a hover-over popup for the rectangles explaining what each thing means to a lay person. I see it at the bottom, but that is below the fold.

Done, I'll take any other suggestions and apply them later, I will also split it a bit for different usecases as this was initially a throwaway prototype but found it useful. Basically it needs a bit more human touch.

Re: DeepSeek V4 Pro 0813

#217

Post by diydsp »

dakolli wrote: post #195
XCSme wrote: post #187

Again, I will wait until there's a provider that doesn't train on prompts before I will benchmark.

psst.. they all do. Also, what kind of IP are you protecting, are you protecting some crazy discovery, nothing you're throwing at them is special, they aren't going to steal your CRUD pomodora app. If anything Deepseek is the only company I'd want to consent to training on my data, they're by far the most altruistic. Atleast they give back all their IP in the form of research and open source weights. It's not like th…

>are you protecting some crazy discovery

Yes. If someone figured out my current project they would have a huge scoop.

Re: DeepSeek V4 Pro 0813

#218

Post by teravor »

npn wrote: post #121
freakynit wrote: post #108

Just tested through openrouter.. gave exactly same task.. the task was to scan existing repo, and generate a single docker-compose file to deploy behind a caddy server, where certain port ranges are already used, the service demands widlcard certificates to be provisioned from outside, and postgre needs to be built-in one... Tested this model, and gpt-5.6-terra-high. Results: this one had few issues. terra: none. The…

wait for Deepseek Harness (yes it is the official name) release then try again. for your kind of task, harness tools matter.

> Deepseek Harness

I wonder if it's just going to be a rebranded Reasonix (https://github.com/esengine/DeepSeek-Reasonix) as it's the only harness their docs mention and it seems specially built for deepseek models.

Re: DeepSeek V4 Pro 0813

#219

Post by gafferongames »

ferongr wrote: post #68
jklmnopqrstuvw wrote: post #65

Tested both DS v4 pro 0813 and Grok 4.6 (all from openrouter) on Codex cli. Worked on a same new feature development on my project. Deepseek 4 pro: Worked for 12m 02s - cost $0.12 - has bug. Grok 4.6: Worked for 3m 18s - cost $ 1.41 - no bug.

[flagged]

Yes.

Re: DeepSeek V4 Pro 0813

#220

Post by throwaway9af11m »

simonw wrote: post #153
wolttam wrote: post #143

Earlier quoted context omitted.

I think I saw a better overall composition out of Flash 0731 Effort on this one?

Default effort for OpenRouter. I'll try a grid of efforts... Wow, the low, medium, and high pelicans came out in surprisingly different styles: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...

[flagged]
Post reply on HN