21–30 of 765 posts
Earlier quoted context omitted.
Is DeepSeek really that big of a deal that everyone else should worry?
Everyone else should rejoice. OpenAI is probably cooked, however. Nvidia might be cooked too.
Initially I was using Claude 3.5 sonnet, then writing unit tests and manually correcting sonnet's code. Sonnet's code mostly worked, except for failing certain complicated combined book updates.
Then I fed the code and the tests into DeepSeek. It turned out pretty bad. At first it tried to make the results of the tests conform to the erroneous results of the code. When I pointed that out, it fixed the immediate logical problem in the code, introducing two more nested problems that we're not there before by corrupting the existing code. After prompted that, it fixed the first error it introduced but left the second one. Then I fixed it myself, uploaded the fix and asked it to summarize what it has done. It started basically gaslighting me, saying that the initial code had the problem that it introduced.
In summary, I lost two days, reverted everything and went back to Sonnet.
For those who aren't tempted to click through, the buried lede for this (and why I'm glad it's being linked to again today) is that "99% of the code in this PR [for llama.cpp] is written by DeekSeek-R1" as conducted by Xuan-Son Nguyen. That seems like a notable milestone.
Yes, but:
"For the qX_K it's more complicated, I would say most of the time I need to re-prompt it 4 to 8 more times.
The most difficult was q6_K, the code never works until I ask it to only optimize one specific part, while leaving the rest intact (so it does not mess up everything)" [0]
And also there:
"You must start your code with #elif defined(__wasm_simd128__)
To think about it, you need to take into account both the refenrence code from ARM NEON and AVX implementation."
[0] https://gist.github.com/ngxson/307140d24d80748bd683b396ba13b...
Earlier quoted context omitted.
Yeah, it is definitely a big deal. I expect it will be a net positive: they proved that you can both train and run inference against powerful models for way less compute than people had previously expected - and they published enough details that other AI labs are already starting to replicate their results. I think this will mean cheaper, faster, and better models. This FAQ about it is very good: https://stratechery…
Why did DeepSeek not kept this for themselves? Is this a Meta style scorched earth strategy?
Personally this looks to me like an ego thing: the DeepSeek team are really, really good and their CEO is enjoying the enormous attention they are getting, plus the pride of proving that Chinese AI labs can take the lead in a field that everyone thought the USA was unassailable in.
Maybe they are true believers in building and sharing "AGI" with the world?
Lots of people see this as a Chinese government backed conspiracy to undermine the US AI industry. I'm not sure how credible that idea is.
I saw somewhere (though I've not confirmed it with a second source) that none of the people listed on the DeepSeek papers got educated at US universities - they all went to school in China, which further emphasizes how good China's home-grown talent pool has got.
This is an overstatement. There are still humans in the loop to do the prompt, apply the patch, verify, write tests, and commit. We're not even at intern-level autonomy here.
> it can optimize its own code This is an overstatement. There are still humans in the loop to do the prompt, apply the patch, verify, write tests, and commit. We're not even at intern-level autonomy here.
I am writing some python code to do Order Flow Imbalance analysis from L2 orderbook updates. The language is unimportant: the logic is pretty subtle, so that the main difficulties are not in the language details, but in the logic and handling edge cases. Initially I was using Claude 3.5 sonnet, then writing unit tests and manually correcting sonnet's code. Sonnet's code mostly worked, except for failing certain compl…
Earlier quoted context omitted.
A lot of the niceness about DeepSeek-R1's usage in coding is that you can see the thought process, which (IME) has been more useful than the final answer. It may well be that o1's chain of thought reasoning trace is also quite good. But they hide it as a trade secret and supposedly ban users for trying to access it, so it's hard to know.
One example from today: I had a coding bug which I asked R1 about. The final answer wasn't correct, but adapting an idea from the CoT trace helped me fix the bug. o1's answer was also incorrect. Interestingly though, R1 struggled in part because it needed the value of some parameters I didn't provide, and instead it made an incorrect assumption about its value. This was apparent in the CoT trace, but the model didn't…
Earlier quoted context omitted.
Yeah, it is definitely a big deal. I expect it will be a net positive: they proved that you can both train and run inference against powerful models for way less compute than people had previously expected - and they published enough details that other AI labs are already starting to replicate their results. I think this will mean cheaper, faster, and better models. This FAQ about it is very good: https://stratechery…
Why did DeepSeek not kept this for themselves? Is this a Meta style scorched earth strategy?
I am writing some python code to do Order Flow Imbalance analysis from L2 orderbook updates. The language is unimportant: the logic is pretty subtle, so that the main difficulties are not in the language details, but in the logic and handling edge cases. Initially I was using Claude 3.5 sonnet, then writing unit tests and manually correcting sonnet's code. Sonnet's code mostly worked, except for failing certain compl…
DeepSeek v3 or DeepSeek R1?