Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

cerebras.ai

Post reply on HN
Print view View original

41–50 of 160 posts

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#41

Post by throwaw12 »

bredren wrote: post #34
throwaw12 wrote: post #31

Earlier quoted context omitted.

Have you used Claude Code and how do you compare the quality to Claude models? I am heavily invested in tools around Claude, still struggling to make a switch and start experimenting with other models

Are you using Claude code or the web interface? I would like to try this with CC myself, apparently with some proxy use an OpenAI compatible LLM can be swapped in.

I am using Claude code, my experience with it so far is great. I use it primarily from terminal, this way I stay focused while reading code and CC doing its job in the background.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#42

Post by bluelightning2k »

This is (slightly) old news from July 8, resurfaced due to the Qwen 3 coder.

I think the gist of this thread is entirely: "please do the same for Qwen 3 coder", with us all hoping for:

a) A viable alternative to Sonnet 3 b) Specifically a faster and cheaper alternative

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#43

Post by smcleod »

aurareturn wrote: post #19

If this is the full fp16 quant, you'd need 2TB of memory to use with the full 131k context. With 44GB of SRAM per Cerebras chip, you'd need 45 chips chained together. $3m per chip. $135m total to run this. For comparison, you can buy a DGX B200 with 8x B200 Blackwell chips and 1.4TB of memory for around $500k. Two systems would give you 2.8TB memory which is enough for this. So $1m vs $135m to run this model. It's no…

There is no reason to run models for inference at static fp16, modern quantisation formats dynamically assign precision to the layers that need them, an average of 6bpw is practical imperceptible from full precision, 8bpw if you really want to squeeze every tiny last drop out of it (although it's unlikely it will be detectable). That is a huge memory saving.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#44

Post by throwawaymaths »

sailingparrot wrote: post #37
aurareturn wrote: post #19

If this is the full fp16 quant, you'd need 2TB of memory to use with the full 131k context. With 44GB of SRAM per Cerebras chip, you'd need 45 chips chained together. $3m per chip. $135m total to run this. For comparison, you can buy a DGX B200 with 8x B200 Blackwell chips and 1.4TB of memory for around $500k. Two systems would give you 2.8TB memory which is enough for this. So $1m vs $135m to run this model. It's no…

> With 44GB of SRAM per Cerebras chip, you'd need 45 chips chained together. $3m per chip. $135m total to run this. That's not how you would do it with Cerebras. 44GB is SRAM, so on chip memory, not HBM memory where you would store most of the params. For reference one GB200 has only 126MB of SRAM, if you tried to estimate how many GB200 you would need for a 2TB model just by looking at the L2 cache size you would ge…

what are the bandwidth/latency of memoryX? those are the key parameters for inference

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#45

Post by throwawaymaths »

zozbot234 wrote: post #38
meowface wrote: post #20

Earlier quoted context omitted.

I just set up Groq with Kimi K2 the other day and was blown away by the speed. Deciding if I should switch to Qwen 3 and Cerebras. (Also, off-topic, but the name reminds me of cerebrates from Starcraft. The Zerg command hierarchy lore was fascinating when I was a young child.)

Obligatory reminder that 'Groq' and 'Grok' are entirely different and unrelated. No risk of a runaway Mecha-Hitler here!

instead risk of requiring racks of hardware to run just one model!

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#46

Post by bredren »

throwaw12 wrote: post #41
bredren wrote: post #34

Earlier quoted context omitted.

Are you using Claude code or the web interface? I would like to try this with CC myself, apparently with some proxy use an OpenAI compatible LLM can be swapped in.

I am using Claude code, my experience with it so far is great. I use it primarily from terminal, this way I stay focused while reading code and CC doing its job in the background.

I’ve heard this repeated that using the env vars you can use gpt models, for example.

But then also that running a proxy tool locally is needed.

I haven’t tried this setup, and can’t say offhand if Cerebras’ hosted qwen described here is “OpenAI” compatible.

I also don’t know if all of the tools CC uses out of the box are supported in the most compatible non-Anthropic models.

Can anyone provide clarity / additional testimony on swapping out the engine on Claude Code?

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#47

Post by throwawaymaths »

pjs_ wrote: post #18

Cerebras is truly one of the maddest technical accomplishments that Silicon Valley has produced in the last decade or so. I met Andy seven or eight years ago and I thought they must have been smoking something - a dinner plate sized chip with six tons of clamping force? They made it real, and in retrospect what they did was incredibly prescient

nah, it was designed for hpc and raw flops. llm inference really requires memory bandwidth.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#48

Post by mehdibl »

meowface wrote: post #20
0vermorrow wrote: post #14

I'm eagerly awaiting for Qwen 3 coder being available on Cerebras. I run plenty of agent loops and the speed makes a somewhat interesting difference in time "compression". Having a Claude 4 Sonnet-level model running at 1000-1500 tok/s would be extremely impressive. To FEEL THE SPEED, you can either try it yourself on Cerebras Inference page, through their API, or for example on Mistral / Le Chat with their "Flash An…

I just set up Groq with Kimi K2 the other day and was blown away by the speed. Deciding if I should switch to Qwen 3 and Cerebras. (Also, off-topic, but the name reminds me of cerebrates from Starcraft. The Zerg command hierarchy lore was fascinating when I was a young child.)

Issue most groq models are limited in context as that cost a lot of memory.
Post reply on HN