Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v3.1

api-docs.deepseek.com

Post reply on HN
Print view View original

251–260 of 273 posts

Re: DeepSeek-v3.1

#251

Post by hangonhn »

Alifatisk wrote: post #203
tw1984 wrote: post #123

just saw this on Chinese internet - deepseek officially mentioned that v3.1 is trained using UE8M0 FP8 as that is the FP8 to be supported by the next gen Chinese AI chip. so basically - some Chinese next gen AI chips is coming, deepseek is working with them to get its flagship model trained using such domestic chips. interesting time ahead! just imagine what it could do to NVIDIA share price when deepseek releases a…

Time to short Nvidia?

No because people never really talk about the quantity of the alternatives -- i.e. Huawei Ascent. Even if Huawei can match the quality, their yields are still abysmal. The numbers I've heard are in the hundreds of thousands vs. millions by Nvidia. In the near future, Nvidia's dominance is pretty secure. The only thing that can threaten it is if this whole AI thing isn't worth what some people imagined it is worth and people start to realize this.

Re: DeepSeek-v3.1

#252

Post by danielhanchen »

linuxftw wrote: post #197
segmondy wrote: post #181

Earlier quoted context omitted.

if you are running a 2bit quant, you are not giving up performance but gaining 100% performance since the alternative is usually 0%. Smaller quants are for folks who won't be able to run anything at all, so you run the largest you can run relative to your hardware. I for instance often ran Q3_K_L, I don't think of how much performance I'm giving up, but rather how without Q3, I won't be able to run it at all. With th…

That's true only in a vacuum. For example, should I run gpt-oss-20b unquantized or gpt-oss-120b quantaized? Some models have a 70b/30b spread, and that's only across a single base model, where many different models exist at different quants could be compared for different tasks.

For GPT OSS in particular, OpenAI only released the MoEs in MXFP4 (4bit), so the "unquantized" version is 4bit MoE + 16bit attention - I uploaded "16bit" versions to https://huggingface.co/unsloth/gpt-oss-120b-GGUF, and they use 65.6GB whilst MXFP4 uses 63GB, so it's not that much difference - same with GPT OSS 20B

llama.cpp also unfortunately cannot quantize matrices that are not a multiple of 256 (2880)

Re: DeepSeek-v3.1

#253

Post by ychan268 »

Alifatisk wrote: post #203
tw1984 wrote: post #123

just saw this on Chinese internet - deepseek officially mentioned that v3.1 is trained using UE8M0 FP8 as that is the FP8 to be supported by the next gen Chinese AI chip. so basically - some Chinese next gen AI chips is coming, deepseek is working with them to get its flagship model trained using such domestic chips. interesting time ahead! just imagine what it could do to NVIDIA share price when deepseek releases a…

Time to short Nvidia?

No evidence v3.1 is trained on Chinese chips(they said very ambiguously, only said they adapted the model for Chinese chips, could be training, could be inference)

Anyway, from my experience, if China really has advanced AI chips for SOTA model, I am sure propaganda machine will go all out, look how they boasted Huawei CPU that’s two generations behind Qualcomm and TSMC

Re: DeepSeek-v3.1

#254

Post by jstummbillig »

segmondy wrote: post #31
hodgehog11 wrote: post #2

For reference, here is the terminal-bench leaderboard: https://www.tbench.ai/leaderboard Looks like it doesn't get close to GPT-5, Claude 4, or GLM-4.5, but still does reasonably well compared to other open weight models. Benchmarks are rarely the full story though, so time will tell how good it is in practice.

garbage benchmark, inconsistent mix of "agent tools" and models. if you wanted to present a meaningful benchmark, the agent tools will stay the same and then we can really compare the models. there are plenty of other benchmarks that disagree with these, with that said. from my experience most of these benchmarks are trash. use the model yourself, apply your own set of problems and see how well it fairs.

Which benchmarks are not garbage?

I don't consider myself super special. I think it should be doable to create a benchmark that beats me having to test every single new model.

Re: DeepSeek-v3.1

#255

Post by cowpig »

danielhanchen wrote: post #83
tw1984 wrote: post #72

Earlier quoted context omitted.

for such dynamic 2bit, is there any benchmark results showing how many performance I would give up compared to the original model? thanks.

Currently no, but I'm running them! Some people on the aider discord are running some benchmarks!

@danielhanchen do you publish the benchmarks you run anywhere?

Re: DeepSeek-v3.1

#256

Post by bobro »

hereme888 wrote: post #230
bobro wrote: post #225

Earlier quoted context omitted.

I’m not asking broadly about difference in ethics. I’m asking specifically about IP theft in the AI space.

I'm not aware of any proven IP theft by American companies in the AI space. Many pending legal challenges. None yet proven.

Alright. So there’s proven IP theft by the Chinese companies with completed legal proceedings?

Re: DeepSeek-v3.1

#257

Post by exe34 »

pshirshov wrote: post #233
exe34 wrote: post #104

Earlier quoted context omitted.

hey fellow crazy person! slight tangent: one thing that helps keep me grounded with "LLMs are doing much more than regurgitation" is watching them try to get things to work on nixos - and hitting every rake on the way to hell! nixos is such a great way to expose code doing things it shouldn't be doing.

In my experience LLMs can do Nix very well, even the models I run locally. I just instruct them to pull dependencies through flake.nix and use direnv to run stuff.

oh yes they do nix very well, but I asked cursor to set up a firecracker vm with networking for exposing a port on the host, and use conda inside to install a certain version of python with some libraries. I asked for a firecracker-vm.nix, a build.sh, a run.sh and a close.sh. it kept trying to run code inside its own fhs-env, which would run, and then when I tried it outside of the fhs, it would fail. I'd paste in the errors and it would without fail say oh let's try the proper nix version of python - which I explicitly didn't want, because I wanted to run conda versions on other machines. I tried to guide it through conda-shell but didn't get very far. in the end I ended up using docker instead, which it did set up without fail.

but when it was failing on my original idea, it kept trying dumb things that weren't really even nix after a while.

Re: DeepSeek-v3.1

#258

Post by green7ea »

pdimitar wrote: post #16
gunalx wrote: post #13

Earlier quoted context omitted.

You would still need drivers and all the stuff difficult with nvidia in linux with a egpu. (Its not nessecarily terrible just suboptimal) Rather just add the second GPU in the Workstation, or just run the llm in your AMD GPU.

Oh, we can run LLMs efficiently with AMD GPUs now? Pretty cool, I haven't been following, thank you.

llama.cpp and lmstudio have a Vulkan backend which is pretty fast. I'm using it to run models on a Strix Halo laptop and it works pretty well.

Re: DeepSeek-v3.1

#259

Post by caycep »

this might be OT and covered somewhere else but what's the latest/greatest on these models and their effect on the linguistics field, vs. what does the latest and greatness in linguistics feel about these models?

Re: DeepSeek-v3.1

#260

Post by esafak »

indigodaddy wrote: post #57
bigyabai wrote: post #6

Earlier quoted context omitted.

Those Qwen3 2507 models are the local creme-de-la-creme right now. If you've got any sort of GPU and ~32gb of RAM to play with, the A3B one is great for pair-programming tasks.

Do we get these good qwen models when using qwen-code CLI tool and authing via qwen.ai account?

You do not need qwen-code or qwen.ai to use them; openrouter + opencode suffice.
Post reply on HN