Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

cerebras.ai

Post reply on HN
Print view View original

31–40 of 160 posts

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#31

Post by throwaw12 »

meowface wrote: post #20
0vermorrow wrote: post #14

I'm eagerly awaiting for Qwen 3 coder being available on Cerebras. I run plenty of agent loops and the speed makes a somewhat interesting difference in time "compression". Having a Claude 4 Sonnet-level model running at 1000-1500 tok/s would be extremely impressive. To FEEL THE SPEED, you can either try it yourself on Cerebras Inference page, through their API, or for example on Mistral / Le Chat with their "Flash An…

I just set up Groq with Kimi K2 the other day and was blown away by the speed. Deciding if I should switch to Qwen 3 and Cerebras. (Also, off-topic, but the name reminds me of cerebrates from Starcraft. The Zerg command hierarchy lore was fascinating when I was a young child.)

Have you used Claude Code and how do you compare the quality to Claude models? I am heavily invested in tools around Claude, still struggling to make a switch and start experimenting with other models

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#32

Post by skeezyboy »

meowface wrote: post #21
mohsen1 wrote: post #16

K2 is also now available on Groq https://console.groq.com/docs/model/moonshotai/kimi-k2-instr... very fun to see agents using those backends

There are rumors that the K2 model Groq is serving is quantized or otherwise produces lower-quality responses than expected due to some optimization, FYI. I tested it and the speed is incredible, though.

have they managed to remove the "output may contain mistakes" disclaimer from a single LLM yet?

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#33

Post by logicchains »

pr337h4m wrote: post #27
TechDebtDevin wrote: post #5

Earlier quoted context omitted.

Its not a new model, but rather their infrastructure and hardware they are showcasing.

Groq appears to have quantized the Kimi K2 model they're serving, which is part of the reason why there's a noticeable performance gap between K2 on Moonshot's official API and the one served by Groq. We don't know how/whether the Qwen3-235B served by Cerebras has been quantized.

Cerebras have previously stated for other models they hosted that they didn't quantise, unlike Groq.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#34

Post by bredren »

throwaw12 wrote: post #31
meowface wrote: post #20

Earlier quoted context omitted.

I just set up Groq with Kimi K2 the other day and was blown away by the speed. Deciding if I should switch to Qwen 3 and Cerebras. (Also, off-topic, but the name reminds me of cerebrates from Starcraft. The Zerg command hierarchy lore was fascinating when I was a young child.)

Have you used Claude Code and how do you compare the quality to Claude models? I am heavily invested in tools around Claude, still struggling to make a switch and start experimenting with other models

Are you using Claude code or the web interface? I would like to try this with CC myself, apparently with some proxy use an OpenAI compatible LLM can be swapped in.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#35

Post by [deleted] »

aurareturn wrote: post #19

If this is the full fp16 quant, you'd need 2TB of memory to use with the full 131k context. With 44GB of SRAM per Cerebras chip, you'd need 45 chips chained together. $3m per chip. $135m total to run this. For comparison, you can buy a DGX B200 with 8x B200 Blackwell chips and 1.4TB of memory for around $500k. Two systems would give you 2.8TB memory which is enough for this. So $1m vs $135m to run this model. It's no…

[deleted]

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#36

Post by thegeomaster »

aurareturn wrote: post #19

If this is the full fp16 quant, you'd need 2TB of memory to use with the full 131k context. With 44GB of SRAM per Cerebras chip, you'd need 45 chips chained together. $3m per chip. $135m total to run this. For comparison, you can buy a DGX B200 with 8x B200 Blackwell chips and 1.4TB of memory for around $500k. Two systems would give you 2.8TB memory which is enough for this. So $1m vs $135m to run this model. It's no…

You're assuming that the whole model has to be in SRAM.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#37

Post by sailingparrot »

aurareturn wrote: post #19

If this is the full fp16 quant, you'd need 2TB of memory to use with the full 131k context. With 44GB of SRAM per Cerebras chip, you'd need 45 chips chained together. $3m per chip. $135m total to run this. For comparison, you can buy a DGX B200 with 8x B200 Blackwell chips and 1.4TB of memory for around $500k. Two systems would give you 2.8TB memory which is enough for this. So $1m vs $135m to run this model. It's no…

> With 44GB of SRAM per Cerebras chip, you'd need 45 chips chained together. $3m per chip. $135m total to run this.

That's not how you would do it with Cerebras. 44GB is SRAM, so on chip memory, not HBM memory where you would store most of the params. For reference one GB200 has only 126MB of SRAM, if you tried to estimate how many GB200 you would need for a 2TB model just by looking at the L2 cache size you would get 16k GB200 aka ~600M$, obviously way off.

Cerebras uses a different architecture than Nvidia, where the HBM is not directly packaged with the chips, this is handled by a different system so you can scale memory and compute separately. Specifically you can use something like MemoryX to act as your HBM which will be high speed interconnected to the chips SRAM, see [1]. I'm not at all an expert in Cerebras, but IIRC you can connect up to like 2PB of memory to a single Cererbas chip, so almost 1000x the FP16 model.

[1]: https://www.cerebras.ai/blog/announcing-the-cerebras-archite...

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#38

Post by zozbot234 »

meowface wrote: post #20
0vermorrow wrote: post #14

I'm eagerly awaiting for Qwen 3 coder being available on Cerebras. I run plenty of agent loops and the speed makes a somewhat interesting difference in time "compression". Having a Claude 4 Sonnet-level model running at 1000-1500 tok/s would be extremely impressive. To FEEL THE SPEED, you can either try it yourself on Cerebras Inference page, through their API, or for example on Mistral / Le Chat with their "Flash An…

I just set up Groq with Kimi K2 the other day and was blown away by the speed. Deciding if I should switch to Qwen 3 and Cerebras. (Also, off-topic, but the name reminds me of cerebrates from Starcraft. The Zerg command hierarchy lore was fascinating when I was a young child.)

Obligatory reminder that 'Groq' and 'Grok' are entirely different and unrelated. No risk of a runaway Mecha-Hitler here!

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#40

Post by stingraycharles »

aurareturn wrote: post #19

If this is the full fp16 quant, you'd need 2TB of memory to use with the full 131k context. With 44GB of SRAM per Cerebras chip, you'd need 45 chips chained together. $3m per chip. $135m total to run this. For comparison, you can buy a DGX B200 with 8x B200 Blackwell chips and 1.4TB of memory for around $500k. Two systems would give you 2.8TB memory which is enough for this. So $1m vs $135m to run this model. It's no…

So, does that mean that in general for the most modern high end LLM tools, to generate ~1500 tokens per seconds you need around $500k in hardware?

Checking: Anthropic charges $70 per 1 million output tokens. @1500 tokens per second that would be around 10 cents per second, or around $8k per day.

The $500k sounds about right then, unless I’m mistaken.

Post reply on HN