I love how Deepseek just casually drops new updates (that deliver big improvements) without fanfare.
I think they did make an announcement on WeChat.
Deepseek R1-0528
51–60 of 264 posts
Re: Deepseek R1-0528
#52Out of sheer curiosity: What’s required for the average Joe to use this, even at a glacial pace, in terms of hardware? Or is it even possible without using smart person magic to append enchanted numbers and make it smaller for us masses?
> even at a glacial pace
If speed is truly not an issue, you can run Deepseek on pretty much any PC with a large enough swap file, at a speed of about one token every 10 minutes assuming a plain old HDD.
Something more reasonable would be a used server CPU with as many memory channels as possible and DDR4 ram for less than $2000.
But before spending big, it might be a good idea to rent a server to get a feel for it.
Re: Deepseek R1-0528
#53No information to be found about it. Hopefully we get benchmarks soon. Reminds me of the days when Mistral would just tweet a torrent magnet link
There's a table here showing some "Overall" and "Median" score, but no context on what exactly was tested. It appears to be in the ballpark as the latest models, but with some cost advantages with the downside of being just as slow as the original r1 (likely lots of thinking tokens). https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd....
Re: Deepseek R1-0528
#54I want to see it in groq asap !
Groq doesn't even have any true deepseek models --- I thought they only had `deepseek-r1-distill-llama-70b` which was distilled onto llama 70b [1].
Re: Deepseek R1-0528
#55Out of sheer curiosity: What’s required for the average Joe to use this, even at a glacial pace, in terms of hardware? Or is it even possible without using smart person magic to append enchanted numbers and make it smaller for us masses?
Hardware: any computer from the last 20 or so years.
Software: client of choice to https://openrouter.ai/deepseek/deepseek-r1-0528
Sorry I'm being cheeky here, but realistically unless you want to shell out 10k for the equivalent of a Mac Studio with 512GB of RAM, you are best using other services or a small distilled model based on this one.
Re: Deepseek R1-0528
#56Earlier quoted context omitted.
On the day Nvidia report earnings too. Pretty sure it's just a coincidence, bro.
Yeah the timing seems strange. Considering how much money will move hands based on those results this might be some kind of play to manipulate the market at least a bit.
Is releasing a better product really "market manipulation"? It seems to me like regular, good competition.
Re: Deepseek R1-0528
#57No information to be found about it. Hopefully we get benchmarks soon. Reminds me of the days when Mistral would just tweet a torrent magnet link
Benchmarks seem like a fools errand at this point; overly tuning models just to specific test already published tests, rather than focusing on making them generalize. Hugging face has a leader board and it seems dominated by models that are finetunings of various common open source models, yet don't seem be broader used: https://huggingface.co/open-llm-leaderboard
The only benchmarks that match my experience with different models are here https://livebench.ai/#/
Re: Deepseek R1-0528
#58No information to be found about it. Hopefully we get benchmarks soon. Reminds me of the days when Mistral would just tweet a torrent magnet link
There's a table here showing some "Overall" and "Median" score, but no context on what exactly was tested. It appears to be in the ballpark as the latest models, but with some cost advantages with the downside of being just as slow as the original r1 (likely lots of thinking tokens). https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd....
It’s appeared on the Livecodebench leaderboard too. Performance on par with O4 Mini - https://livecodebench.github.io/leaderboard.html
Re: Deepseek R1-0528
#59No information to be found about it. Hopefully we get benchmarks soon. Reminds me of the days when Mistral would just tweet a torrent magnet link
Benchmarks seem like a fools errand at this point; overly tuning models just to specific test already published tests, rather than focusing on making them generalize. Hugging face has a leader board and it seems dominated by models that are finetunings of various common open source models, yet don't seem be broader used: https://huggingface.co/open-llm-leaderboard
There are quite a few benchmarks for which that's not the case:
- live benchmarks (livebench, livecodebench, matharena, SWE-rebench, etc)
- benchmarks that do not have a fixed structure, like games or human feedback benches (balrog, videogamebench, arena)
- (to some extent) benchmark without existing/published answers (putnambench, frontiermath). You could argue that someone could hire people to solve those or pay off benchmark dev, but it's much more complicated.
Most of the benchmarks that don't try to tackle future contamination are much less useful, that's true. Unfortunately, HLE kind of ignored it (they plan to add a hidden set to test for contamination, but once the answers are there, it's a lost game IMHO); I really liked the concept.
Edit: it is true that these benchmarks are focusing only on a fairly specific subset of the model capabilities. For everything else vibe check is your best bet.
Re: Deepseek R1-0528
#60Earlier quoted context omitted.
Yeah the timing seems strange. Considering how much money will move hands based on those results this might be some kind of play to manipulate the market at least a bit.
Is releasing a better product really "market manipulation"? It seems to me like regular, good competition.
It's "manipulating the market" only when your geopolitical adversary brings the competition.