Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek V4 Pro 0813

openrouter.ai

Post reply on HN
Print view View original

191–200 of 493 posts

Re: DeepSeek V4 Pro 0813

#191

Post by ApolloFortyNine »

spijdar wrote: post #130
svachalek wrote: post #91

Earlier quoted context omitted.

Maybe it's me but I don't see how DS Flash is better than GLM at all, much less by a huge gap. I'd probably protest less against Fable and Opus being put at the same level than many would, but there's no denying the two models are a very different experience from each other. I guess where I'm going is no one should pick a model by the benchmarks.

I'm not the most LLM-savvy person around, and I'm not gonna say I've put a ton of effort into practically compared these open models. But, a month or two ago I did do some "practical evaluates" testing GLM 5.2 versus DSv4 (flash/pro) with OpenCode's subscription with some late 80s Unix clone-type work, and this jives with my experience. GLM ended up being far slower, and far more expensive, for approximately the same…

My experience is the same.

Imo it has a lot to do with you/the harness tries to get it to test itself. Deepseek v4 flash seems more than capable of understanding when something has failed, and making changes until it works. I've definitely seen it make mistakes I would expect something like Opus to find, but it works through them on it's own (and for literal pennies).

At the end of the day, I think that's one of the most important features of a model.

Re: DeepSeek V4 Pro 0813

#192

Post by tanishqkanc »

bob1029 wrote: post #169
derangedHorse wrote: post #126

Earlier quoted context omitted.

Terra has not been able to do any of the technical tasks I've asked of it correctly. I'm surprised others get use out of it. Anything below Sol high tends to give me mostly unreliable results. I'm using codex as my main harness but maybe it performs better with a different one.

I am seeing essentially deterministic results with Terra running a custom browser automation agent across >100 interaction events. The harness is everything. If I just threw something like Codex at this and said "good luck" I wouldn't make it beyond 5-10 interactions. I tried that already. Carefully designing the views and tools over the environment is where you can go from 50% to 99.9999%.

Curious about what you found. I agree harness for browser automation is vital - I work on https://libretto.sh

Re: DeepSeek V4 Pro 0813

#193

Post by Phemist »

cyanydeez wrote: post #144
derangedHorse wrote: post #126

Earlier quoted context omitted.

Terra has not been able to do any of the technical tasks I've asked of it correctly. I'm surprised others get use out of it. Anything below Sol high tends to give me mostly unreliable results. I'm using codex as my main harness but maybe it performs better with a different one.

the breadth and width of the universe of oneshot challenges are all arbitrary. It's unsurprising different workflows oneshot better than others. All the more reason to favor local models under your control, as once you find that sweet spot model, no one can change it, upgrade it, align it, take it down or otherwise harm the time investment you made it making it your own. I can't really believe no one understands, aft…

Exactly! I am not opposed to cloud-based models, but I do only stick to open-weight models because I know I can move my whole stack to local (given enough hardware) and continue development without any of the LLM interaction contracts being broken.

I would like to see some development where proof of authenticity certs are generated alongside the actual output of the model. Prove to me (or at least claim to me liable to breach of contract) that this output was generated by FP8 DeepSeek V4 Pro 0813. Not some cheaper quantization of the model.

Re: DeepSeek V4 Pro 0813

#194

Post by zeven7 »

derangedHorse wrote: post #126
freakynit wrote: post #108

Just tested through openrouter.. gave exactly same task.. the task was to scan existing repo, and generate a single docker-compose file to deploy behind a caddy server, where certain port ranges are already used, the service demands widlcard certificates to be provisioned from outside, and postgre needs to be built-in one... Tested this model, and gpt-5.6-terra-high. Results: this one had few issues. terra: none. The…

Terra has not been able to do any of the technical tasks I've asked of it correctly. I'm surprised others get use out of it. Anything below Sol high tends to give me mostly unreliable results. I'm using codex as my main harness but maybe it performs better with a different one.

I bounce between Sol high/medium and Luna max. I don't know why you'd use anything between Luna max and Sol medium. Luna is so extremely cheap and cranked up to max it does anything I'd want Terra to do for a fraction of the cost. What is Terra for?

Re: DeepSeek V4 Pro 0813

#195

Post by dakolli »

XCSme wrote: post #187

Again, I will wait until there's a provider that doesn't train on prompts before I will benchmark.

psst.. they all do. Also, what kind of IP are you protecting, are you protecting some crazy discovery, nothing you're throwing at them is special, they aren't going to steal your CRUD pomodora app.

If anything Deepseek is the only company I'd want to consent to training on my data, they're by far the most altruistic. Atleast they give back all their IP in the form of research and open source weights. It's not like they're hoarding your data for them to make money, they're basically giving everything out for free. The only reason you even have the option of waiting for another provider is because they release weights.

They're releasing all their IP, which is a trillion times more valuable than anything you're providing, you people are just greedy and oddly self centered.

Re: DeepSeek V4 Pro 0813

#196

Post by yassa9 »

freakynit wrote: post #108

Just tested through openrouter.. gave exactly same task.. the task was to scan existing repo, and generate a single docker-compose file to deploy behind a caddy server, where certain port ranges are already used, the service demands widlcard certificates to be provisioned from outside, and postgre needs to be built-in one... Tested this model, and gpt-5.6-terra-high. Results: this one had few issues. terra: none. The…

did you test kimi k3 or qwen 3.8 max on the same task ? or plan to test them ? I respect those genuine users tests other than those benchmarks that models are trained and overfitted to them

Re: DeepSeek V4 Pro 0813

#197

Post by v3ss0n »

freakynit wrote: post #108

Just tested through openrouter.. gave exactly same task.. the task was to scan existing repo, and generate a single docker-compose file to deploy behind a caddy server, where certain port ranges are already used, the service demands widlcard certificates to be provisioned from outside, and postgre needs to be built-in one... Tested this model, and gpt-5.6-terra-high. Results: this one had few issues. terra: none. The…

I do that kind of things all the time with Qwen 3.5 122B. It works well in one shot with Cline or Opencode.

May be your harness problem?

Re: DeepSeek V4 Pro 0813

#198

Post by bigmadshoe »

jklmnopqrstuvw wrote: post #65

Tested both DS v4 pro 0813 and Grok 4.6 (all from openrouter) on Codex cli. Worked on a same new feature development on my project. Deepseek 4 pro: Worked for 12m 02s - cost $0.12 - has bug. Grok 4.6: Worked for 3m 18s - cost $ 1.41 - no bug.

Why are people giving these n=1 comparisons like they mean anything? The worst offender is that pelican guy. These are non-deterministic systems and a single trial should not update your priors much at all.

Of course it's significant that your response had a bug and took four times longer, but if you're only going to try once, this isn't real science, it's just vibes.

Re: DeepSeek V4 Pro 0813

#199

Post by amelius »

freakynit wrote: post #108

Just tested through openrouter.. gave exactly same task.. the task was to scan existing repo, and generate a single docker-compose file to deploy behind a caddy server, where certain port ranges are already used, the service demands widlcard certificates to be provisioned from outside, and postgre needs to be built-in one... Tested this model, and gpt-5.6-terra-high. Results: this one had few issues. terra: none. The…

I didn't understand your use case, so it could also be the way you write your prompt, I suppose ...

Re: DeepSeek V4 Pro 0813

#200

Post by XCSme »

dakolli wrote: post #195
XCSme wrote: post #187

Again, I will wait until there's a provider that doesn't train on prompts before I will benchmark.

psst.. they all do. Also, what kind of IP are you protecting, are you protecting some crazy discovery, nothing you're throwing at them is special, they aren't going to steal your CRUD pomodora app. If anything Deepseek is the only company I'd want to consent to training on my data, they're by far the most altruistic. Atleast they give back all their IP in the form of research and open source weights. It's not like th…

I "trust" what they say on OpenRouter for the provider, for some it says they retain prompts, for other that they retain but can also use them for training.

It's not any crazy IP, just my own benchmarks/tests, once they are in the training set it defeats the purpose of the tests, and I have to make new ones.

Post reply on HN