Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

news.ycombinator.com

Post reply on HN
Print view View original

151–160 of 214 posts

Re: DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

#151

Post by zamadatix »

ycui7 wrote: post #147
aftbit wrote: post #67

>In keeping with our commitment to user responsibility, following the official launch of V4.1 Flash and prior to the release of V4.1 Pro, all requests to the Pro model will be routed to V4.1 Flash and billed at Flash's price Please don't do this kind of thing. If a user has validated a workflow on V4 Pro, they might not want to suddenly start testing it in production on V4.1 Flash. Instead, keep V4 Pro around but dep…

llm is not a deterministic program. the same model won't even return deterministic answer. what's the point keep the model freezed? if you want deterministic returns, you should set the temperature to 0 to get the best possibility of deterministic.

Strict determinism is a different, but related, issue.

E.g. if I've written a role playing character using a specific model I may want to pin the character to that model until I've been able to test the model being "better" doesn't affect the feel of the character before switching. That doesn't mean I need the character's responses to be completely deterministic, but that doesn't imply I'm fine with the character having a different quality or feel of response just because the new model is out.

It'd be nice if there was a more explicit way to signal in the request "I want what you think is best per dollar for this class of answer" vs "I want this model to answer".

Re: DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

#152

Post by nolok »

idiotsecant wrote: post #110
nolok wrote: post #103

Earlier quoted context omitted.

Usually I would very much agree with you, but those things are not deterministic so if that's an issue for you you're probably not making the right choices.

Models are not deterministic, but they do have a flavor . When that flavor changes it can change the nature of output in a way that is undesirable. Sort of like shooting a rifle - where the bullets hit is (to some order of magnitude, no philosophizing please) non-deterministic, but different very similar rifles will group differently and need to be appropriately adjusted to hit anything.

Yeah but then it doesn't matter if it's "the model changed to another one" or "the model changed but it's the same name". Point is, you're not hosting it, as far as you know it can change at any moment, build around that idea. Is that great no, is that ideal no, that's why I self host (I include actually renting online the capacity and hosting the model myself on it).

Re: DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

#153

Post by lkois »

nolok wrote: post #103
aftbit wrote: post #67

>In keeping with our commitment to user responsibility, following the official launch of V4.1 Flash and prior to the release of V4.1 Pro, all requests to the Pro model will be routed to V4.1 Flash and billed at Flash's price Please don't do this kind of thing. If a user has validated a workflow on V4 Pro, they might not want to suddenly start testing it in production on V4.1 Flash. Instead, keep V4 Pro around but dep…

Usually I would very much agree with you, but those things are not deterministic so if that's an issue for you you're probably not making the right choices.

That's a narrow take. Non-deterministic doesn't mean random; workflows can be reasonably validated and consistent to some known degree.

I work for an education department that serves a chatbot for students, and model changes go through painstaking content safety reviews. I initially assumed it's just a bunch of bureaucratic paranoia. But every other model upgrade has a measurably different adherence to the existing system prompts about not talking to the kids about sex and drugs and mental health issues.

Re: DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

#154

Post by badatnames »

simonw wrote: post #49

> all requests to the Pro model will be routed to V4.1 Flash and billed at Flash's price If I'd carefully tested and optimized prompts against Pro I wouldn't be keen on this particular news. I feel like API model providers should lean towards not swapping out models on their paying customers, no matter how much "better" the new model is meant to be.

I imagine they're doing this due to capacity issues or somesuch. They can always relaunch Pro later, meanwhile a little ricered benchmaxxing of their existing flash model provides a temporary cover story. They certainly aren't silly enough to think this won't impact existing Pro users

Re: DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

#155

Post by nolok »

kamranjon wrote: post #141
packetlost wrote: post #112

Earlier quoted context omitted.

> They’re nondeterministic at a fine level, but can be “deterministic” at a more general level: e.g. you might know that one model will always return properly formatted json when asked. That might not be true of the replacement, even if it is in general “better” and cheaper. This isn't true. Even Sol messes up JSON formatting for me on occasion. Do not delude yourself into thinking these things are reliable. They are…

Is nobody using structured outputs? They use constrained decoding at the generation stage to ensure the probability of tokens that would break the format are set to 0. I kinda figured everyone was doing this at this point.

If you use a large enough volume, you will know this isn't fully reliable. You might get json, and it might not match what the model actually sent because the last layer cut it up to match what you want. At the end, not json, or json but not really matching what the model wanted, it's sort of the same issue: when you use them you HAVE to assume they can have a brain fart. That's fine, just code around it.

Re: DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

#156

Post by tarruda »

kamranjon wrote: post #100
tarruda wrote: post #18

Earlier quoted context omitted.

IQ3_XXS (~3.2 BPW). For me this is an option because my Mac studio is only used for serving LLMs, so I can afford to dedicate most of its RAM to this. I can run with 256k context and only uses ~117G, with the remaining (up to 125G which I can allocate to VRAM) being used for prompt caching and context checkpoints. I'm making my own quants, though the Vision-Exp version is outdated and won't work on llama.cpp master b…

Hey there! I do the same but I use dwarfstar at a 2-bit quant: https://github.com/antirez/ds4 I'm curious if you've tried dwarfstar and decided to move to llama.cpp and 3 bit quants or what made you go that route instead? I've been using ds4 for months now and it's already got support for the new vision model, haven't tried it yet, still on 0731 but it's been very solid for me.

I tried dwarfstar when llama.cpp DSV4 support was still very weak, and while it worked, I didn't see anything that would make me want to stick with it vs llama.cpp. llama.cpp is simply better with its awesome built-in webui, router and server APIs and certainly support much more models and quantizations than dwarfstar.

Since then, I started maintaining my own vibe coded dsv4 branch with metal optimizations, so I actually get much better metal performance on my llama.cpp branch than on dwarfstar (plus all the extra llama.cpp features). Here it is in case you want to give it a shot: https://github.com/tarruda/llama.cpp/tree/qwen4exp-dsv4-opti...

Re: DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

#157

Post by WhyNotHugo »

nolok wrote: post #103
aftbit wrote: post #67

>In keeping with our commitment to user responsibility, following the official launch of V4.1 Flash and prior to the release of V4.1 Pro, all requests to the Pro model will be routed to V4.1 Flash and billed at Flash's price Please don't do this kind of thing. If a user has validated a workflow on V4 Pro, they might not want to suddenly start testing it in production on V4.1 Flash. Instead, keep V4 Pro around but dep…

Usually I would very much agree with you, but those things are not deterministic so if that's an issue for you you're probably not making the right choices.

A dice roll is non-deterministic.

Replacing a six-sided die for an eight-sided die also keeps rolls non-deterministic.

That doesn't mean it's fine to just replace the dice mid-game.

Re: DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

#158

Post by rpdillon »

CharlesW wrote: post #62
mattmcal wrote: post #57

Earlier quoted context omitted.

It's not just web chat, V4 Flash 7/31 suffers from a lot of pathological behavior in coding harnesses as well, e.g. infinite loops, hallucinations, premature termination, and invalid tool calls.

FWIW, I haven’t experienced any of that using V4 Flash via DeepSeek in omp. What’s your coding harness and inference provider?

Same. My side projects are coded almost exclusively with the Deepseek V4 Flash 07/31 in omp, and it recovers beautifully in every case. I'm using OpenCode Zen.

Re: DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

#159

Post by wongarsu »

kamranjon wrote: post #141
packetlost wrote: post #112

Earlier quoted context omitted.

> They’re nondeterministic at a fine level, but can be “deterministic” at a more general level: e.g. you might know that one model will always return properly formatted json when asked. That might not be true of the replacement, even if it is in general “better” and cheaper. This isn't true. Even Sol messes up JSON formatting for me on occasion. Do not delude yourself into thinking these things are reliable. They are…

Is nobody using structured outputs? They use constrained decoding at the generation stage to ensure the probability of tokens that would break the format are set to 0. I kinda figured everyone was doing this at this point.

Most serious providers are now supporting structured outputs in a reasonable way for all model configs. But for example on ollama structured outputs are still incompatible with tool calling and with reasoning

Re: DeepSeek launching v4.1 flash cheaper and more capable than v4 pro

#160

Post by disiplus »

WhyNotHugo wrote: post #157
nolok wrote: post #103

Earlier quoted context omitted.

Usually I would very much agree with you, but those things are not deterministic so if that's an issue for you you're probably not making the right choices.

A dice roll is non-deterministic. Replacing a six-sided die for an eight-sided die also keeps rolls non-deterministic. That doesn't mean it's fine to just replace the dice mid-game.

Sure, but they are replacing same generation model to another not just switching opus to gpt.
Post reply on HN