Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek V4 – almost on the frontier

simonwillison.net

Post reply on HN
Print view View original

241–250 of 420 posts

Re: DeepSeek V4 – almost on the frontier

#241

Post by _0ffh »

scrollop wrote: post #231
cedws wrote: post #113

The biggest differentiator for me: DeepSeek just does what I ask. I've tried using both GPT and Claude for reverse engineering recently, both refused. I even got a warning on my OpenAI account.

Obscene levels of hallucinations, the worst of LLMs, unfortunately. Deepseek v4 pro 94% Deepseek v4 flash - 96% https://artificialanalysis.ai/evaluations/omniscience?models...

Personally, I'm not bothered very much by LLM confabulation, as long as it's the result of missing context. In most practical tasks, we either give context to the model, or tell it to find it itself using the internet. What I am concerned with is confabulation that contradicts available in-context information, but that doesn't seem to be what is measured here.

Re: DeepSeek V4 – almost on the frontier

#242

Post by logicprog »

baldai wrote: post #143
gmerc wrote: post #44

Earlier quoted context omitted.

Speak for yourself. I found switching from Opus 4.7 to be completely painless and in fact, due to the reliability of Anthropic’s API, less of a friction despite slower response times. Zero issues on a large mono repro

Hi, I am happy it works well for you. For me personally I struggle finding good use-cases in general for these OOS models. I am lightly technical but I do not manually code. So my flow is /grill-me (can take hours), make plan, review plan with 2. model, implement, review after implementation. Maybe it is because my tasks are usually chunkier, or because I cant code myself that I struggle using cheaper models. Feels l…

In my experience, deep seek models are massively overrated in terms of how good they actually are at agantic usage, coding and writing, just because they are kind of the first open source entrant and the name a lot of people know. Try GLM 5.1, coding and writing just because they are kind of the first open source entrant and the name a lot of people know. Try GLM 5.1.

Re: DeepSeek V4 – almost on the frontier

#243

Post by 0xkvyb »

It might be at the frontier, but DeepSeek is really struggling with compute. The amount of 429 Rate Limit responses I've been getting just testing this thing made me pause all my attempts at cross-comparing it to others.

I'm gonna stick to GLM5.1 for now.

Re: DeepSeek V4 – almost on the frontier

#244

Post by 0xkvyb »

kamikazechaser wrote: post #230
cedws wrote: post #113

The biggest differentiator for me: DeepSeek just does what I ask. I've tried using both GPT and Claude for reverse engineering recently, both refused. I even got a warning on my OpenAI account.

In my experience GLM 5.1 has been excellent when paired with IDA Pro (DeepSeek v4 pro comes in close second, Kimi straight up refuses). Claude can only do reverse engineering if you throw it into some sort of hero/saviour mode then gradually pivot into red team (though it gets easily tripped).

Yes, GLM 5.1 is surprisingly good! Particularly for long-horizon Agentic tasks, with 100+ available tools. It really shocked me in a good way when it was able to complete a long run with 50+ steps and not fall into a loop along the way.

Re: DeepSeek V4 – almost on the frontier

#245

Post by ryandrake »

estearum wrote: post #236
ryandrake wrote: post #217

Earlier quoted context omitted.

> I even got a warning on my OpenAI account. This idea of software threatening the user with consequences is totally wild and dystopian. Fellow developers, what kind of world have be built? This is insanity. Imagine if my hammer told me, "Hey, you shouldn't use me on screws--only nails. Do it again and I'll self-destruct!" WTF people, stop making this kind of software!

All sorts of tools try to prevent dangerous/destructive uses In fact probably every single piece of commercial software you use had you sign a contract saying you wouldn’t do it

> All sorts of tools try to prevent dangerous/destructive uses

But they don't threaten their users or have an "N strikes and you're out" policy. I take those safety caps off of all the chemicals in my garage because I'm a grown-ass adult and those caps are a pain in the butt. I would not expect the manufacturer of a solvent to show up at my house lecturing me about safety and threatening to ban me from buying his products.

Re: DeepSeek V4 – almost on the frontier

#246

Post by indigodaddy »

shlewis wrote: post #172
MintsJohn wrote: post #104

Earlier quoted context omitted.

Glm5.1 is fantastic for me. But that could be how I use it, I don't ask it to build entire apps or entire features, instead asking it to build piecemeal functionality. For that it compares very well to chatgpt 5.4 (I haven't extensively tried 5.5, it might be better, might be same). I have given deepseekv4 pro a try but not much more than a try, as it performed subpar on 4 tasks in a row (missing the obvious/intended…

I've never asked LLMs to build a whole app without detailed directions. I've done giving it a general data flow, structs and methods..etc Are frontier models capable of building something only with general directions now?

Since about Jan of this year, yes

Re: DeepSeek V4 – almost on the frontier

#247

Post by dust42 »

scrollop wrote: post #231
cedws wrote: post #113

The biggest differentiator for me: DeepSeek just does what I ask. I've tried using both GPT and Claude for reverse engineering recently, both refused. I even got a warning on my OpenAI account.

Obscene levels of hallucinations, the worst of LLMs, unfortunately. Deepseek v4 pro 94% Deepseek v4 flash - 96% https://artificialanalysis.ai/evaluations/omniscience?models...

The output of any LLM is always 100% hallucination by principle. On top of that, most benchmarks are at best an approximation of LLM quality. Your use case decides which one to use. That said, I haven't tested v4 yet but the old 3.2 is still a decent model. And concerning use cases, I had coding problems that Opus couldn't solve but a local 35B model did.

All the talk about frontier and SOTA is do dig deeper and deeper into the pockets of VCs and finally do an IPO.

Re: DeepSeek V4 – almost on the frontier

#248

Post by UlisesAC4 »

scrollop wrote: post #231
cedws wrote: post #113

The biggest differentiator for me: DeepSeek just does what I ask. I've tried using both GPT and Claude for reverse engineering recently, both refused. I even got a warning on my OpenAI account.

Obscene levels of hallucinations, the worst of LLMs, unfortunately. Deepseek v4 pro 94% Deepseek v4 flash - 96% https://artificialanalysis.ai/evaluations/omniscience?models...

This must be easily benchmaxed because I have never gotten an "idk like" answer for the western frontier models. All my personal "real world" use cases will always resort to hallucinations.

Re: DeepSeek V4 – almost on the frontier

#249

Post by dust42 »

antirez wrote: post #238

Related: live demo of DeepSeek v4 Flash running on my 128GB MacBook. Italian language with English subs. https://www.youtube.com/watch?v=todMmp6AGCE

For many models the performance of llama.cpp on Mac is 20-40% lower than MLX. Did you try MLX? At least on HF there are MLX 2-bit quants. Unfortunately I have only 64GB, so I can't test it.

Re: DeepSeek V4 – almost on the frontier

#250

Post by antirez »

dust42 wrote: post #249
antirez wrote: post #238

Related: live demo of DeepSeek v4 Flash running on my 128GB MacBook. Italian language with English subs. https://www.youtube.com/watch?v=todMmp6AGCE

For many models the performance of llama.cpp on Mac is 20-40% lower than MLX. Did you try MLX? At least on HF there are MLX 2-bit quants. Unfortunately I have only 64GB, so I can't test it.

I'm not using llama.cpp there, it's my inference engine that is DeepSeek v4 specific. The goal is to optimize it as much as possible.
Post reply on HN