81–90 of 160 posts
Edit: Looks like it did. They both introduced pay as you go, and have prepaid limits too at $1500. I wonder if they have any limitations on parallel execution for pay as you go...
What would the energy use be for an average query be, when using large models at this speed?
I’ve asked that question on linked in to the Cerebras team a couple times and haven’t ever received a response. There is system max tdp values posted online but I’m not sure you can assume the system is running in max tdp for these queries. If it is the numbers are quite high (I just tried to find the number but couldn’t find it but I had it in my notes as 23kw). If someone from Cerebras is reading this feel free to…
I'm eagerly awaiting for Qwen 3 coder being available on Cerebras. I run plenty of agent loops and the speed makes a somewhat interesting difference in time "compression". Having a Claude 4 Sonnet-level model running at 1000-1500 tok/s would be extremely impressive. To FEEL THE SPEED, you can either try it yourself on Cerebras Inference page, through their API, or for example on Mistral / Le Chat with their "Flash An…
Exactly. I can see my efficiency going up a ton with this kind of speed. Every time I'm waiting for agents my mind looses some focus and context. Running parallel agents gets more speed but at the cost of focus. Near instant iteration loops in Cursor would feel magical (even more magical?). It will also impact how we work: interactive IDEs like Cursor probably make more sense than CLI tools like Claude code when answ…
It opens up a whole lot of use cases that'd be a nightmare if you have to look at each individual change.
Earlier quoted context omitted.
Small bit of pedantry: While there are 1 and 2-bit quantized types used in some aggressive schemes, they aren't floating point so it's inaccurate to preface them with FP. They are int types. The smallest real floating point type is FP4. EDIT: Who knew that correctness is controversial. What a weird place HN has become.
I wonder if the fact that we use "floating point" is itself a bottleneck that can be improved. Remembering my CS classes, storing an FP value requires the base and the exponent; that's a design decision. Also remembering some assembler classes, Int arithmetic is way faster than FP. Could there be a better "representation " for the numbers needed in NN that would provide the accuracy of floating point but provide fast…
A modern processor can do something similar to an integer bit shift about as quickly with a floating point, courtesy of FSCALE instructions and similes. Indeed, modern processors are extremely performant at floating point math.
Earlier quoted context omitted.
Small bit of pedantry: While there are 1 and 2-bit quantized types used in some aggressive schemes, they aren't floating point so it's inaccurate to preface them with FP. They are int types. The smallest real floating point type is FP4. EDIT: Who knew that correctness is controversial. What a weird place HN has become.
I wonder if the fact that we use "floating point" is itself a bottleneck that can be improved. Remembering my CS classes, storing an FP value requires the base and the exponent; that's a design decision. Also remembering some assembler classes, Int arithmetic is way faster than FP. Could there be a better "representation " for the numbers needed in NN that would provide the accuracy of floating point but provide fast…
Yes. Look up “block floating point”.
If this is the full fp16 quant, you'd need 2TB of memory to use with the full 131k context. With 44GB of SRAM per Cerebras chip, you'd need 45 chips chained together. $3m per chip. $135m total to run this. For comparison, you can buy a DGX B200 with 8x B200 Blackwell chips and 1.4TB of memory for around $500k. Two systems would give you 2.8TB memory which is enough for this. So $1m vs $135m to run this model. It's no…
That on-chip SRAM memory is purely temporary working memory and does need to hold the entire model weights. The Cerebras chip works on a sparse weights representation, streams non-zero off their external memory server and the cores work in a transport-triggered dataflow manner.
Earlier quoted context omitted.
have they managed to remove the "output may contain mistakes" disclaimer from a single LLM yet?
Never will. But then, same for humans yes?
Earlier quoted context omitted.
I initially thought this was about the Qwen release from two days ago, Qwen3-235B-A22B-Instruct-2507 - https://simonwillison.net/2025/Jul/22/qwen3-235b-a22b-instru... - but that's a no-reasoning model and the Cerebras announcement talks about reasoning, which tipped me off that this was Qwen's Qwen3-235B-A22B from April. (These model names are so confusing.)
Is Qwen3 235B A22B in OpenRouter the stock version or Cerebras version? https://openrouter.ai/provider/cerebras
Cerebras is truly one of the maddest technical accomplishments that Silicon Valley has produced in the last decade or so. I met Andy seven or eight years ago and I thought they must have been smoking something - a dinner plate sized chip with six tons of clamping force? They made it real, and in retrospect what they did was incredibly prescient
The concept is super cool but does anyone actually use them instead of just buying Nvidia?
isn't Qwen Alibaba's family of models? What does cerebras have to do with this? i'm lost.