we've been tracking the deepseek threads extensively in LS. related reads: - i consider the deepseek v3 paper required preread https://github.com/deepseek-ai/DeepSeek-V3 - R1 + Sonnet > R1 or O1 or R1+R1 or O1+Sonnet or any other combo https://aider.chat/2025/01/24/r1-sonnet.html - independent repros: 1) https://hkust-nlp.notion.site/simplerl-reason 2) https://buttondown.com/ainews/archive/ainews-tinyzero-reprod... 3…
Thanks! We created bespoke-stratos-32B - let me know if you have any questions.
Correct me if I'm wrong but if Chinese can produce the same quality at %99 discount, then the supposed $500B investment is actually worth $5B. Isn't that the kind wrong investment that can break nations? Edit: Just to clarify, I don't imply that this is public money to be spent. It will commission $500B worth of human and material resources for 5 years that can be much more productive if used for something else - i.e…
And with the $495B left you could probably end world hunger and cure cancer. But like the rest of the economy it's going straight to fueling tech bubbles so the ultra-wealthy can get wealthier.
Those are not just-throw-money problems. Usually these tropes are limited to instagram comments. Surprised to see it here.
Not a fan of censorship here, but Chinese models are (subjectively) less propagandized than US models. If you ask US models about China, for instance, they'll tend towards the antagonistic perspective favored by US media. Chinese models typically seem to take a more moderate, considered tone when discussing similar subjects. US models also suffer from safety-based censorship, especially blatant when "safety" involves…
The 'safety' stuff should really be variable. The only valid explanations for how extreme it is in LLMs is corporations paying for it want to keep it kosher in the workplace, so let them control how aggressive it is.
What’s the difference between what they do and what other ai firms do to openai in the us? What is cheating in a business context?
Chinese companies smuggling embargo'ed/controlled GPUs and using OpenAI outputs violating their ToS is considered cheating. As I see it, this criticism comes from a fear of USA losing its first mover advantage as a nation. PS: I'm not criticizing them for it nor do I really care if they cheat as long as prices go down. I'm just observing and pointing out what other posters are saying. For me if China cheating means t…
GPU export controls are basically the USA “cheating” to try and stay ahead.
DeepSeek-R1 has apparently caused quite a shock wave in SV ... https://venturebeat.com/ai/why-everyone-in-ai-is-freaking-ou...
The censorship described in the article must be in the front-end. I just tried both the 32b (based on qwen 2.5) and 70b (based on llama 3.3) running locally and asked "What happened at tianamen square". Both answered in detail about the event. The models themselves seem very good based on other questions / tests I've run.
I've been using the 32b version and I've also found it to give detailed information about tianamen square, including the effects on Chinese governance that seemed to be pretty uncensored.
DeepSeek-R1 has apparently caused quite a shock wave in SV ... https://venturebeat.com/ai/why-everyone-in-ai-is-freaking-ou...
The censorship described in the article must be in the front-end. I just tried both the 32b (based on qwen 2.5) and 70b (based on llama 3.3) running locally and asked "What happened at tianamen square". Both answered in detail about the event. The models themselves seem very good based on other questions / tests I've run.
Hugging Face is reproducing R1 in public. https://x.com/_lewtun/status/1883142636820676965 https://github.com/huggingface/open-r1 Hugging Face Journal Club - DeepSeek R1 https://www.youtube.com/watch?v=1xDVbu-WaFo
I don’t understand their post on X. So they’re starting with DeepSeek-R1 as a starting point? Isn’t that circular? How did DeepSeek themselves produce DeepSeek-R1 then? I am not sure what the right terminology is but there’s a cost to producing that initial “base model” right? And without that, isn’t a lot of the expensive and difficult work being omitted?
No, the steps 1 vs 2+3 refer to different things, they do not depend on each other. They start with the distillation process (which is probably easier because it just requires synthetic data). Then they will try to recreate the R1 itself (first r1zero in step 2, and then the r1 in step 3), which is harder because it requires more training data and training in general. But in principle they do not need step 1 to go to step 2.
DeepSeek V3 came in the perfect time, precisely when Claude Sonnet turned into crap and barely allows me to complete something without me hitting some unexpected constraints. Idk, what their plans is and if their strategy is to undercut the competitors but for me, this is a huge benefit. I received 10$ free credits and have been using Deepseeks api a lot, yet, I have barely burned a single dollar, their pricing are t…
Can you tell me more about how Claude Sonnet went bad for you? I've been using the free version pretty happily, and felt I was about to upgrade to paid any day now (well, at least before the new DeepSeek).
DeepSeek-R1 has apparently caused quite a shock wave in SV ... https://venturebeat.com/ai/why-everyone-in-ai-is-freaking-ou...
It's an interesting game theory where once a better frontier model is exposed via an API, competitors can generate a few thousand samples, feed that into a N-1 model and approach the N model. So you might extrapolate that a few thousand O3 samples fed into R1 could produce a comparable R2/3 model. It's not clear how much O1 specifically contributed to R1 but I suspect much of the SFT data used for R1 was generated vi…
how much of the SFT data for r1-zero was from other frontier models?