I genuinely do not understand the evaluations of the US AI industry. The chinese models are so close and far cheaper
[dead]
DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]
421–430 of 485 posts
Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]
#422Earlier quoted context omitted.
Third party providers are still cheap though. The closed models are the ones where you can't see the real cost to running them.
Oh, I was mostly talking about the Chinese taxpayer footing the training bill. You are right that we can directly observe the cost of inference for open models.
Not sure the Chinese taxpayer is footing the bill though - of course, it might not be net zero, there might be secondary effects, etc.
A few days ago I read an article saying the Chinese utilities have a pricing structure that favors high-tech industries (say, an AI data center), making the difference by charging more the energy-intensive but less sophisticated industries (an aluminium smelter, for example).
Admittedly, there are some advantages when you do central and long-term economic planning.
Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]
#423Disclaimer: I did not test this yet. I don't want to make big generalizations. But one thing I noticed with chinese models, especially Kimi, is that it does very well on benchmarks, but fails on vibe testing. It feels a little bit over-fitting to the benchmark and less to the use cases. I hope it's not the same here.
Weird, I have gone local for the last 2 years. I use Chinese models 90% of the time, Kimi K2 Thinking, DeepSeekv3.Terminus, Qwen3 and GLM4.6. I'm not vibe testing it but really putting them to use and they do keep up great.
Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]
#424How capable are these models at tool calling?
Depends on if they are trained for tool calling, this model is experimentation with new architecture, training methods, etc. It's not designed for tool calling. If you want to tool call, then you should look into DeepSeekv3.1-Terminus.
Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]
#425Earlier quoted context omitted.
Where does the software come from? Your iPhone can’t magically intercept communications and send it to China without the embedded software. If Apple can’t verify the integrity of its operating system before it is installed on iPhones. There are some huge issues. Even if China did manage to embed software on the iPhone in Taiwan, it would soon hopefully be wiped since you usually end up updating the OS anyway as soon…
The hardware can always contain undetectable sub-devices that can magically intercept anything with no possibility for the software to detect this. You should remember that all iPhones had for several years an undetected hardware backdoor, until a couple of years ago, when independent researchers have found it and reported the Apple bugs as CVEs, so Apple was forced to fix the vulnerabilities. The hardware backdoor c…
The argument was that you couldn’t trust American designed hardware running American designed software because it was built in China. All theories suggest that the security vulnerabilities were caused by Apple and had nothing to do with Chinese manufacturers
Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]
#426Earlier quoted context omitted.
Oh they need control of models to be able to censor and ensure whatever happens inside the country with AI stays under their control. But the open-source part? Idk I think they do it to mess with the US investment and for the typical open source reasons of companies: community, marketing, etc. But tbh especially the messing with the US, as a european with no serious competitor, I can get behind.
This is the rare earth minerals dumping all over again. Devalue to such a price as to make the market participants quit, so they can later have a strategic stranglehold on the supply. This is using open source in a bit of different spirit than the hacker ethos, and I am not sure how I feel about it. It is a kind of cheat on the fair market but at the same time it is also costly to China and its capital costs may beco…
Where exactly is this fair market? Giant US companies love rules and regulations, but only when it benefits them (and they pay dearly for it)
Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]
#427Earlier quoted context omitted.
How did we come to the place that the most transparent and open models are now coming out of China—freely sharing their research and source code—while all the American ones are fully locked down
China needs to build the world's trust and respect, while the US is slowly but surely losing theirs.
slowly?
Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]
#428Earlier quoted context omitted.
If you're trying to build AI based applications you can and should compare the costs between vendor based solutions and hosting open models with your own hardware. On the hardware side you can run some benchmarks on the hardware (or use other people's benchmarks) and get an idea of the tokens/second you can get from the machine. Normalize this for your usage pattern (and do your best to implement batch processing whe…
Well the seemingly cheap comes with significantly degraded performance, particular for agentic use. Have you tried replacing Claude Code with some locally deployed model, say, on 4090 or 5090? I have. It is not usable.
Strictly speaking, you have not deployed any model on a 5090 because a 5090 card has never been produced.
And without specifying your quantization level it's hard to know what you mean by "not usable"
Anyway if you really wanted to try cheap distilled/quantized models locally you would be using used v100 Teslas and not 4 year old single chip gaming GPUs.
Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]
#429Earlier quoted context omitted.
Anthropic has RLed the shit out of their models to the extent that they give sub-par answers to general purpose questions. Google has great models but is institutionally incapable of building a cohesive product experience. They are literally shipping their org chart with Gemini (mediocre product), AI Overview (trash), AI Mode (outstanding but limited modality), Gemini for Google Workspace (steaming pile), Gemini on A…
RLed?
Reinforcement learning, I believe
Re: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]
#430So DSA means a lightweight indexing model evaluated over the entire context window + a top-k attention evaluation. There’s no soft max in the indexing model, so it can run blazingly fast in parallel.
I’m surprised that a fixed size k doesn’t experience degrading performance in long context windows though. That’s a _lot_ of responsibility to push into that indexing function. How could such a simple model achieve high enough precision and recall in a fixed size k for long context windows?