DeepSeek-V3-0324 released, 641GB, MIT licensed, >20tok/SEC on $<10k hardware
1–7 of 7 posts
Re: DeepSeek-V3-0324 released, 641GB, MIT licensed, >20tok/SEC on $<10k hardware
#2[flagged]
Re: DeepSeek-V3-0324 released, 641GB, MIT licensed, >20tok/SEC on $<10k hardware
#3Impressive use of reasoner CoT distillation method applied to deepseek R1. MIT license for the weights. Thanks, Deepseek!
Re: DeepSeek-V3-0324 released, 641GB, MIT licensed, >20tok/SEC on $<10k hardware
#4[flagged]
[deleted]
Re: DeepSeek-V3-0324 released, 641GB, MIT licensed, >20tok/SEC on $<10k hardware
#5the non-archive.org submission, where simonw actually is watching and commenting https://news.ycombinator.com/item?id=43462317
Re: DeepSeek-V3-0324 released, 641GB, MIT licensed, >20tok/SEC on $<10k hardware
#6The headline seems to imply that the full 641GB model is running at >20tok/sec on the Mac Studio, but the blog says:
>The model only came out a few hours ago and MLX developer Awni Hannun already has it running at >20 tokens/second on a 512GB M3 Ultra Mac Studio ($9,499 of ostensibly consumer-grade hardware) via mlx-lm and this mlx-community/DeepSeek-V3-0324-4bit 4bit quantization, which reduces the on-disk size to 352 GB.
Re: DeepSeek-V3-0324 released, 641GB, MIT licensed, >20tok/SEC on $<10k hardware
#7The only open AI in town