Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepseek R1-0528

huggingface.co

Post reply on HN
Print view View original

91–100 of 264 posts

Re: Deepseek R1-0528

#91

Post by ToValueFunfetti »

behnamoh wrote: post #88
jazzyjackson wrote: post #75

Earlier quoted context omitted.

No sign of what source material it was trained on though right? So open weight rather than reproducible from source. I remember there's a project "Open R1" that last I checked was working on gathering their own list of training material, looks active but not sure how far along they've gotten: https://github.com/huggingface/open-r1

> No sign of what source material it was trained on though right? out of curiosity, does anyone do anything "useful" with that knowledge? it's not like people can just randomly train models..

Would be useful for answering "is this novel or was it in the training data", but that's not typically what the point of open source is

Re: Deepseek R1-0528

#92

Post by ribelo »

lossolo wrote: post #57
chvid wrote: post #49

Earlier quoted context omitted.

Benchmarks seem like a fools errand at this point; overly tuning models just to specific test already published tests, rather than focusing on making them generalize. Hugging face has a leader board and it seems dominated by models that are finetunings of various common open source models, yet don't seem be broader used: https://huggingface.co/open-llm-leaderboard

The only benchmarks that match my experience with different models are here https://livebench.ai/#/

livebench was good, but now it's a joke. Gemini flash is better in coding than pro and sonnet 3.7. And this is only the beginning of weird results.

Re: Deepseek R1-0528

#93

Post by boroboro4 »

behnamoh wrote: post #86
sergiotapia wrote: post #69

Earlier quoted context omitted.

the only reason they are fast is because the models they host are severely quantized so i've heard.

they responded to my tweet last year and said they didn't quantize the models.

It's very hard to find right now but I'm sure they said they don't quantize KV cache, but their weights are in fp8.

Re: Deepseek R1-0528

#94

Post by stavros »

behnamoh wrote: post #87
fragmede wrote: post #77

Earlier quoted context omitted.

It's. not. open. source! https://www.downloadableisnotopensource.org/

it's got more 'source' than whatever OpenAI provides for their models.

No it doesn't, it has exactly the same source, zero. It has more downloadable binary.

Re: Deepseek R1-0528

#96

Post by halyconWays »

chvid wrote: post #49
_lvbh wrote: post #10

No information to be found about it. Hopefully we get benchmarks soon. Reminds me of the days when Mistral would just tweet a torrent magnet link

Benchmarks seem like a fools errand at this point; overly tuning models just to specific test already published tests, rather than focusing on making them generalize. Hugging face has a leader board and it seems dominated by models that are finetunings of various common open source models, yet don't seem be broader used: https://huggingface.co/open-llm-leaderboard

>overly tuning models just to specific test already published tests, rather than focusing on making them generalize.

I think you just described SATs and other standardized tests

Re: Deepseek R1-0528

#97

Post by boroboro4 »

jbentley1 wrote: post #89
jacob019 wrote: post #65

Earlier quoted context omitted.

Groq has a weak selection of models, which is frustrating because their inference speed is insane. I get it though, selection + optimization = performance.

From conversation with someone from Groq, they have a custom compiler and runtime for the models to run on their custom hardware, which is why the selection is poor. For every model type they need to port the architecture to run on their compiler beforehand.

They can't host DeepSeek because it's too big. Their chips have 230mb of memory, so it will take them ~3000 chips to host the model + (possible large) number of chips to keep kv cache. I bet it's just too hard to bring such topology online at all, and impossible to make even near to be profitable.

Re: Deepseek R1-0528

#98

Post by girvo »

omneity wrote: post #66
lodovic wrote: post #42

Earlier quoted context omitted.

If you use the excess memory for AI only it's cheaper to rent . A single H100 costs less than $2 per hour. (incl power)

Worth mentioning that a single H100 (80-96GB) is not enough to run R1. You're looking at 6-8 GPUs on the lower end, and factor in the setup and download time. An alternative is to use serverless GPU or LLM providers which abstract some of this for you, albeit at a higher cost and slow starts when you first use your model for some time.

It is enough to run the dynamically quantised 1.56 bit version I believe, which is fun to play around with.

Re: Deepseek R1-0528

#100

Post by vietvu »

overfeed wrote: post #72
htrp wrote: post #39

You're gonna need at least 8 h100 80s for this....

That's about $16-24 per hour - depending on the number of tokens you're slinging in that period, it may be much cheaper than paying OpenAI for similar functionality.

Or paying deepseek for slightly cheaper and worse performance than OpenAI.
Post reply on HN