How does DeepSeek work: An inside look
codedoodles.substack.com
How does DeepSeek work: An inside look
1–10 of 45 posts
Re: How does DeepSeek work: An inside look
#2"DeepSeek’s policy states that it stores the information for 'further training' of the chatbot in Chinese servers. While it’s not something to get panicked about (most of the applications follow the same principle, despite not being overly open about it)"
Is this really true?
Re: How does DeepSeek work: An inside look
#3"DeepSeek’s policy states that it stores the information for 'further training' of the chatbot in Chinese servers. While it’s not something to get panicked about (most of the applications follow the same principle, despite not being overly open about it)" Is this really true?
Why wouldn't it be? OpenAI and Anthropic keep everyone's prompts and use them for training too
Re: How does DeepSeek work: An inside look
#4"DeepSeek’s policy states that it stores the information for 'further training' of the chatbot in Chinese servers. While it’s not something to get panicked about (most of the applications follow the same principle, despite not being overly open about it)" Is this really true?
Why wouldn't it be? OpenAI and Anthropic keep everyone's prompts and use them for training too
Anthropic says
> To date we have not used any customer or user-submitted data to train our generative models.
https://www.anthropic.com/news/claude-3-5-sonnet
There's an obvious problem with the concept of training on user prompts; how would training on a bunch of questions cause it to know the answers?
Re: How does DeepSeek work: An inside look
#5"DeepSeek’s policy states that it stores the information for 'further training' of the chatbot in Chinese servers. While it’s not something to get panicked about (most of the applications follow the same principle, despite not being overly open about it)" Is this really true?
Why wouldn't it be? OpenAI and Anthropic keep everyone's prompts and use them for training too
Anthropic states that they don't train on the inputs and outputs of their commercial offerings unless you explicitly opt-in: https://privacy.anthropic.com/en/articles/7996868-i-want-to-...
Do you think they're lying or where you speaking about free tier offerings?
Re: How does DeepSeek work: An inside look
#6Earlier quoted context omitted.
Why wouldn't it be? OpenAI and Anthropic keep everyone's prompts and use them for training too
Anthropic says > To date we have not used any customer or user-submitted data to train our generative models. https://www.anthropic.com/news/claude-3-5-sonnet There's an obvious problem with the concept of training on user prompts; how would training on a bunch of questions cause it to know the answers?
User queries were at least historically useful to train smaller models from larger models. You need to know the kind of questions real people ask to train a model that’s good at answering those questions
Re: How does DeepSeek work: An inside look
#7The headline says "an in-depth look", but the whole post is quite short and doesn't go into much detail. I found these overviews better:
https://www.lesswrong.com/posts/a9GR7m4nyBsqjjL8d/deepseek-r...
https://newsletter.languagemodels.co/p/the-illustrated-deeps...
Re: How does DeepSeek work: An inside look
#8Earlier quoted context omitted.
Why wouldn't it be? OpenAI and Anthropic keep everyone's prompts and use them for training too
Anthropic says > To date we have not used any customer or user-submitted data to train our generative models. https://www.anthropic.com/news/claude-3-5-sonnet There's an obvious problem with the concept of training on user prompts; how would training on a bunch of questions cause it to know the answers?
"There's an obvious problem with the concept of training on user prompts; how would training on a bunch of questions cause it to know the answers?"
I imagine by analysing the chat? If the user says thanks in the end, or gives a thumps up, it likely was a useful and correct answer, that could be included in further training. Or at least considered for future training and I cannot imagine them not considering and experimenting with it.
Re: How does DeepSeek work: An inside look
#9if you know Chinese，this may help you。
https://www.meoai.net/deepseek-r1.html
Re: How does DeepSeek work: An inside look
#10Earlier quoted context omitted.
Why wouldn't it be? OpenAI and Anthropic keep everyone's prompts and use them for training too
Anthropic says > To date we have not used any customer or user-submitted data to train our generative models. https://www.anthropic.com/news/claude-3-5-sonnet There's an obvious problem with the concept of training on user prompts; how would training on a bunch of questions cause it to know the answers?
Back when I started using LLMs for writing code I would type out long, gently phrased explanations about why it was wrong, as if I was teaching a pupil, hoping it would help. I'm sure a lot of us did. If they can parse and mine those prompts, they'll have a nice little metacorpus to build on.
Now I just tell it to stop being stupid over and over until it does a good job. I wonder if it would improve the model to keep all of the beratement in the training data.
Edit: Apparently a 'metacorpus' is a swollen nematode ass. My sincerest apologies, bros.