191–200 of 225 posts
At these prices, you can start throwing Flash at a lot of small, repetitive tasks where you wouldn't even consider using a bigger model before. It feels like the interesting shift is not “Flash replaces Pro”, but “there are now a lot more things worth automating.”
Earlier quoted context omitted.
Which versions of flash and at what thinking levels? Which chinese flash models and at what thinking levels? What tasks? What completion rates? How was quality evaluated?
- Which versions: 3.6 vs 3.7 vs. 3.8 for Gemini Flash, and v4 0731 for Deepseek v4 Flash, and GLM 5.3 Flash - Medium for Gemini, high for Deepseek. - Things like find information, then understand something about it, then send a slack message or email etc. - Completion rates somewhere in 80-90%, Deepseek a bit better than Gemini - Quality evaluated by Fable 5.1 and Astra 6.0 acting as a rubric judge. Gemini quality wo…
I hope DeepSeek takes some time to improve their tuning for reasoning effort. Right now, there are only three reasoning efforts: low, high, and max. For all intents and purposes, "low" is pretty much the same as turning reasoning off, and "high" is similar to "max". "High/max" performs way too much reasoning, takes forever, and causes costs to balloon. They need a proper "medium" setting. I get it that they're probab…
I switched to GLM-5.3 flash on high for this reason. Too many "but wait" in the Deepseek-v4 reasoning.
I just wish they kept parameter count down in order to fit entirely within commonly used RAM sizes
Earlier quoted context omitted.
I finally uninstalled the app yesterday after giving it plenty of chances over several months. Yesterday, I asked it whether «DeepSeek has fixed the issue where it erroneously answers in Chinese?» and it answered in Chinese.
So you did not do what the post you replied to suggested?
>In keeping with our commitment to user responsibility, following the official launch of V4.1 Flash and prior to the release of V4.1 Pro, all requests to the Pro model will be routed to V4.1 Flash and billed at Flash's price Please don't do this kind of thing. If a user has validated a workflow on V4 Pro, they might not want to suddenly start testing it in production on V4.1 Flash. Instead, keep V4 Pro around but dep…
Usually I would very much agree with you, but those things are not deterministic so if that's an issue for you you're probably not making the right choices.
Earlier quoted context omitted.
Models are not deterministic, but they do have a flavor . When that flavor changes it can change the nature of output in a way that is undesirable. Sort of like shooting a rifle - where the bullets hit is (to some order of magnitude, no philosophizing please) non-deterministic, but different very similar rifles will group differently and need to be appropriately adjusted to hit anything.
Yeah but then it doesn't matter if it's "the model changed to another one" or "the model changed but it's the same name". Point is, you're not hosting it, as far as you know it can change at any moment, build around that idea. Is that great no, is that ideal no, that's why I self host (I include actually renting online the capacity and hosting the model myself on it).
Earlier quoted context omitted.
It's very easy to tell who is not running production applications using these models based on comments like this
Let me give you the benefit of the doubt, can you expand what you run in production?
Sounds nice! But, the web ui chat version of flash has very poor language following abilities in my experience: You may ask it something in English, and get a thinking chain in Chinese with an answer in Chinese, or an English thinking chain and an English answer. Using the retry button on the same question has a 50/50 chance of any of those results. Sometimes, asking something in English, but where information are mo…
Earlier quoted context omitted.
All of these flash models have this. You have to build your harness so that it deals with it. Infinite loops are solved by having an error message that says what to do differently on failure, invalid tool calls are solved by making the tool schema less strict and detect things in the runtime etc. Hallucinations you can't fix. Gemini is a bit worse there than DeepSeek, but there's not much research on how to fix that.…
Can you share what tools and processes you're using to do this? I've been using Pi to build custom extensions and wrapping workflows in shell processes to make it more deterministic and enforce certain validations, all guided by Fable. This isn't production work, though, just playing llm factorio at home.
You replay all your sessions against your harness, and then store all logs all output, everything to a safe place.
Finally use a blind judge to check everything, and score the output.
Then fix your harness, iterate again until better until you are in a point where it's just the model's weakness. If you get to that, use a bigger model.
> all requests to the Pro model will be routed to V4.1 Flash and billed at Flash's price If I'd carefully tested and optimized prompts against Pro I wouldn't be keen on this particular news. I feel like API model providers should lean towards not swapping out models on their paying customers, no matter how much "better" the new model is meant to be.
Seems to be lots of people in here worried about “if i had”, and nobody who actually has done this. Anybody actually using deepseek in a production system affected by this want to share their experience?