Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

cerebras.ai

Post reply on HN
Print view View original

21–30 of 160 posts

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#21

Post by meowface »

mohsen1 wrote: post #16

K2 is also now available on Groq https://console.groq.com/docs/model/moonshotai/kimi-k2-instr... very fun to see agents using those backends

There are rumors that the K2 model Groq is serving is quantized or otherwise produces lower-quality responses than expected due to some optimization, FYI.

I tested it and the speed is incredible, though.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#23

Post by logicchains »

0vermorrow wrote: post #14

I'm eagerly awaiting for Qwen 3 coder being available on Cerebras. I run plenty of agent loops and the speed makes a somewhat interesting difference in time "compression". Having a Claude 4 Sonnet-level model running at 1000-1500 tok/s would be extremely impressive. To FEEL THE SPEED, you can either try it yourself on Cerebras Inference page, through their API, or for example on Mistral / Le Chat with their "Flash An…

It'll be nice if this generates more pressure on programming language compilation times. If agentic LLMs get fast enough that compilation time becomes the main blocker in the development process, there'll be significant economic incentives for improving compiler performance.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#24

Post by cherryteastain »

pjs_ wrote: post #18

Cerebras is truly one of the maddest technical accomplishments that Silicon Valley has produced in the last decade or so. I met Andy seven or eight years ago and I thought they must have been smoking something - a dinner plate sized chip with six tons of clamping force? They made it real, and in retrospect what they did was incredibly prescient

The concept is super cool but does anyone actually use them instead of just buying Nvidia?

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#26

Post by vFunct »

pjs_ wrote: post #18

Cerebras is truly one of the maddest technical accomplishments that Silicon Valley has produced in the last decade or so. I met Andy seven or eight years ago and I thought they must have been smoking something - a dinner plate sized chip with six tons of clamping force? They made it real, and in retrospect what they did was incredibly prescient

Wafer-scale integration was done decades before.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#27

Post by pr337h4m »

TechDebtDevin wrote: post #5
pr337h4m wrote: post #2

Quantization?

Its not a new model, but rather their infrastructure and hardware they are showcasing.

Groq appears to have quantized the Kimi K2 model they're serving, which is part of the reason why there's a noticeable performance gap between K2 on Moonshot's official API and the one served by Groq.

We don't know how/whether the Qwen3-235B served by Cerebras has been quantized.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#28

Post by scottcha »

rsolva wrote: post #22

What would the energy use be for an average query be, when using large models at this speed?

I’ve asked that question on linked in to the Cerebras team a couple times and haven’t ever received a response. There is system max tdp values posted online but I’m not sure you can assume the system is running in max tdp for these queries. If it is the numbers are quite high (I just tried to find the number but couldn’t find it but I had it in my notes as 23kw).

If someone from Cerebras is reading this feel free to dm me as optimizing this power is what we do.

Re: Cerebras launches Qwen3-235B, achieving 1.5k tokens per second

#29

Post by lordofgibbons »

aurareturn wrote: post #19

If this is the full fp16 quant, you'd need 2TB of memory to use with the full 131k context. With 44GB of SRAM per Cerebras chip, you'd need 45 chips chained together. $3m per chip. $135m total to run this. For comparison, you can buy a DGX B200 with 8x B200 Blackwell chips and 1.4TB of memory for around $500k. Two systems would give you 2.8TB memory which is enough for this. So $1m vs $135m to run this model. It's no…

Almost everyone runs LLM inference at fp8 - for all of the open models anyway. You only see performance drop off below fp8.
Post reply on HN