Live data from Hacker News

DeepSeek-V4-Flash means LLM steering is interesting again

seangoedecke.com

41–50 of 84 posts

Re: DeepSeek-V4-Flash means LLM steering is interesting again

#41
post #21

Earlier quoted context omitted.

That's not what people mean when they talk about censoring. They mean that models are trained to not touch some subjects, and that can spill over in legit tasks, often with humorous results (early on, there were many instances of models refusing to answer "how do you kill a process", because of overbearing refusal training). Uncensoring a model also doesn't necessarily improve generic use cases. In fact it can lead t…

Anthropic mentioned explicitly making an effort to make Opus 4.7 worse at cybersecurity tasks because the last few generations have been getting too good at them. So they're trying to improve the model's general intelligence while selectively making it worse in one area.

It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter. [1]

I think that the best use of frontier AI models outside of generic corporate settings is going to be building generic frameworks and procedures for training specialized models. No ethically-trained American coding model would ever consent to write a Plutonium Process Engineering agent. But you can get it to write a general framework for pretraining models and preparing them for agentic usage, to which the copious published literature on plutonium production could be given as a data set.

[1] https://blog.codinghorror.com/your-favorite-programming-quot...

Re: DeepSeek-V4-Flash means LLM steering is interesting again

#42

Earlier quoted context omitted.

M2.7 is no longer open source, it's been changed to a NC license. It's an OK model, but IME out of the big 5 chinese models (ds, glm, kimi, minimax and qwen), DS models have generally shown better generalisation and real-world usage than all the others, even if the benchmark scores were lower. Less benchmaxxxing, basically. DS4 also has some neat new arch improvements, giving it a lot of context at lower VRAM usage.…

M2.7 was never open source, only open weight, which fulfills a lot of the spirit of open source, but isn't really the same thing as a whole. The noncommercial license is basically impossible to enforce if you're self-hosting anyway, because it's essentially impossible to prove that any individual commit was made by Minimax M2.7 in an environment where multiple self-hosted models are being run side-by-side. Besides th…

KV cache size is the main constraint on batching (for any given ctx length), that's a huge deal for efficiency both locally and in the data center. DeepSeek V4's reduced KV requirement is a real game changer, it definitively unlocks batching requests together for local inference, not just at scale.

Re: DeepSeek-V4-Flash means LLM steering is interesting again

#43
This reminds me of control vectors, especially this line in the linked DwarfStar repo:

> y = y - scale * direction[layer] * dot(direction[layer], y)

From https://vgel.me/posts/representation-engineering/

> A control vector is a vector (technically a list of vectors, one per layer) that you can apply to model activations during inference to control the model's behavior without additional prompting

Re: DeepSeek-V4-Flash means LLM steering is interesting again

#44
post #28

Earlier quoted context omitted.

> Uncensoring a model also doesn't necessarily improve generic use cases. While the following is not a generic use case, I have a funny anecdote about how censorship is holding back flagship models. I was asking an uncensored version of Qwen3.6 how a CLI option of llama.cpp worked, and to my horror and amazement, it rudely went and decompiled the binary to figure it out. It felt like the computer-equivalent of asking…

> Flagship models usually do not do that without some convincing Just a data point, but I’ve been having Claude do this regularly

I think I was using GitHub Copilot when I made the experience that led me to this statement. I guess the experience of using LLMs can be quite different depending on model version and harness.

Re: DeepSeek-V4-Flash means LLM steering is interesting again

#45

I know it's only tangentially relevant, but I've been baffled by the interest in DeepSeek V4 Flash. It's larger, less efficient, and in many cases, performs worse on both objective benchmarks and real world sniff test (admittedly, n=1) than Minimax M2.7. DS4F hallucinates at extraordinary rates while M2.7 does not. The 196k context length that M2.7 was natively trained up represents neither a hard technical ceiling (…

May I ask you what did you used for the DS4F inference? It is a model with very low hallucination rate in my tests.

Re: DeepSeek-V4-Flash means LLM steering is interesting again

#46
post #16

Earlier quoted context omitted.

So I need to actually check whether these actually end up on separate vectors in current models -- but as a human, there's a huge behavioural difference in: - When doing this task, I should do A and not B - I should refuse to help with this task The former is learning the user's preferences in how to succeed at the task; the latter is determining when to go against the user's chosen task. Your example: - "Are vaccine…

Does DeepSeek V4 actually refuse the latter task? As I mentioned, I find it to be very light on refusals already.

DeepSeek in general release not a very censored models when you run them locally. E.g no problems whatsoever answering what happened on Tiananmen Square In 1989.

Re: DeepSeek-V4-Flash means LLM steering is interesting again

#47
post #45

I know it's only tangentially relevant, but I've been baffled by the interest in DeepSeek V4 Flash. It's larger, less efficient, and in many cases, performs worse on both objective benchmarks and real world sniff test (admittedly, n=1) than Minimax M2.7. DS4F hallucinates at extraordinary rates while M2.7 does not. The 196k context length that M2.7 was natively trained up represents neither a hard technical ceiling (…

May I ask you what did you used for the DS4F inference? It is a model with very low hallucination rate in my tests.

Per AA's Omniscience Index benchmark, the "non-hallucination rate" subcomponent (1 - hallucination rate) of 4% for DS4F vs 66% for M2.7.

https://artificialanalysis.ai/leaderboards/models?weights=op...

Re: DeepSeek-V4-Flash means LLM steering is interesting again

#48

Earlier quoted context omitted.

M2.7 is no longer open source, it's been changed to a NC license. It's an OK model, but IME out of the big 5 chinese models (ds, glm, kimi, minimax and qwen), DS models have generally shown better generalisation and real-world usage than all the others, even if the benchmark scores were lower. Less benchmaxxxing, basically. DS4 also has some neat new arch improvements, giving it a lot of context at lower VRAM usage.…

M2.7 was never open source, only open weight, which fulfills a lot of the spirit of open source, but isn't really the same thing as a whole. The noncommercial license is basically impossible to enforce if you're self-hosting anyway, because it's essentially impossible to prove that any individual commit was made by Minimax M2.7 in an environment where multiple self-hosted models are being run side-by-side. Besides th…

As to the 2nd part of your message, it's really easy to verify yourself (on openrouter).

DSv4-flash is currently being served at 0.14/0.24 $/MTok by most of the providers (8 as of writing this) and even a bit cheaper by 2 providers.

Minimax2.7 is being served at 0.30/1.20 $/MTok by most providers (4 providers as of writing this) and double that price by 2 providers.

As for the first part of your message, this is actually a good illustration of the miss-understanding of licensing LLMs. There are open-source models out there (Apache 2.0 and MIT) and there are also source-available (i.e. open weights) in llamas, minimax2.7 and something in between with the latest kimi (MIT w/ attribution). Open source in the context of LLMs means that you get a license to run, inspect, modify and re-release a model. It was never about data or training. But that's a very common interpretation, that's wrong IMO. But I get that it's contested, so anyway. Sorry for the tangent.

Re: DeepSeek-V4-Flash means LLM steering is interesting again

#49

Earlier quoted context omitted.

M2.7 was never open source, only open weight, which fulfills a lot of the spirit of open source, but isn't really the same thing as a whole. The noncommercial license is basically impossible to enforce if you're self-hosting anyway, because it's essentially impossible to prove that any individual commit was made by Minimax M2.7 in an environment where multiple self-hosted models are being run side-by-side. Besides th…

KV cache size is the main constraint on batching (for any given ctx length), that's a huge deal for efficiency both locally and in the data center. DeepSeek V4's reduced KV requirement is a real game changer, it definitively unlocks batching requests together for local inference, not just at scale.

This may be relevant for parallelizable workloads. For reference on my perspective: I come at this as someone who is exclusively concerned with sequential, non-parallelizable, single-user, single-system workloads.

Re: DeepSeek-V4-Flash means LLM steering is interesting again

#50

Earlier quoted context omitted.

not sure why youre fixed on censoring. if we invert your POV censoring includes not reporting falsehoods "vaccines are harmful". Science and logic often tackle these subject via censoring, but a model given a equal sampling of Internet, would think vacinnes are harmful. a less naive correction would censor this problematic context. so im cofised as to why you think unmasking whatever bias you think is censored will r…

[flagged]

[deleted]
Post reply on HN