Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras achieves 2,500T/s on Llama 4 Maverick (400B)

cerebras.ai

Post reply on HN
Print view View original

51–60 of 100 posts

Re: Cerebras achieves 2,500T/s on Llama 4 Maverick (400B)

#51

Post by K0balt »

subscribed wrote: post #45
aktuel wrote: post #43

Earlier quoted context omitted.

Some quant-heads endorsing the latest fad doesn't prove anything. Also they don't care if chinese hackers are vacuuming data cause ballstreet doesn't care about sustainability. But I grant you that secure and trust are just words that don't mean anything anymore anyhow.

LOL, all fintech are using or entering the "cloud" very heavily. Cloud is here for long enough that claiming it's insecure shows only the immense ignorance.

Any business using commercial inference providers is potentially risking their value proposition. Everything you send to cloud inference will eventually be gleaned for training data.

Empirically we know that the data is the most valuable input to cloud services, and eventually it will be used, regardless of the user agreement. When the stored data becomes worth more than the company, it will be eaten and stripped by vulture capital. Law of the jungle, baby.

Re: Cerebras achieves 2,500T/s on Llama 4 Maverick (400B)

#52

Post by arresin »

kristianp wrote: post #32
iansinnott wrote: post #31

Earlier quoted context omitted.

Although not obvious, you _can_ pay them per token. You have to use OpenRouter or Huggingface as the inference API provider. https://cerebras-inference.help.usepylon.com/articles/192554...

Interestingly, llama 4 maverick isn't available on that page, only scout.

Yeap looks like it’s just scout and lower.

Re: Cerebras achieves 2,500T/s on Llama 4 Maverick (400B)

#53

Post by arresin »

twothreeone wrote: post #17
turblety wrote: post #8

Maybe one day they’ll have an actual api that you can pay per token. Right now it’s the standard “talk to us” if you want to use it.

Huh? Just make an account, get your API key, and try out the free tier.. works for me. https://cloud.cerebras.ai

> that you can pay per token

Re: Cerebras achieves 2,500T/s on Llama 4 Maverick (400B)

#55

Post by x-complexity »

ryao wrote: post #3

> At over 2,500 t/s, Cerebras has set a world record for LLM inference speed on the 400B parameter Llama 4 Maverick model, the largest and most powerful in the Llama 4 family. This is incorrect. The unreleased Llama 4 Behemoth is the largest and most powerful in the Llama 4 family. As for the speed record, it seems important to keep it in context. That comparison is only for performance on 1 query, but it is well kno…

Pretty much no disagreements IMO.

By the time the CSE-5 is rolled out, it *needs* at least 500GB of SRAM to make it worthwhile. Multi-layer wafer stacking's the only path to advance this chip.

Re: Cerebras achieves 2,500T/s on Llama 4 Maverick (400B)

#57

Post by jenny91 »

bubblethink wrote: post #12
ryao wrote: post #3

> At over 2,500 t/s, Cerebras has set a world record for LLM inference speed on the 400B parameter Llama 4 Maverick model, the largest and most powerful in the Llama 4 family. This is incorrect. The unreleased Llama 4 Behemoth is the largest and most powerful in the Llama 4 family. As for the speed record, it seems important to keep it in context. That comparison is only for performance on 1 query, but it is well kno…

>Each one costs ~$2 million, so that is $40 million. Pricing for exotic hardware that is not manufactured at scale is quite meaningless. They are selling tokens over an API. The token pricing is competitive with other token APIs.

I agree on the first. On the second: I would bet a lot of money that they aren't actually breaking even on their API (or even close to). They don't have a "pay as you go" per-token tier, it's all geared up to demonstrate use of their API as a novelty. They're probably burning cash on every single token. But their valuation and hype has surely gone way up since they got onto LLMs.

Re: Cerebras achieves 2,500T/s on Llama 4 Maverick (400B)

#58

Post by bubblethink »

jenny91 wrote: post #57
bubblethink wrote: post #12

Earlier quoted context omitted.

>Each one costs ~$2 million, so that is $40 million. Pricing for exotic hardware that is not manufactured at scale is quite meaningless. They are selling tokens over an API. The token pricing is competitive with other token APIs.

I agree on the first. On the second: I would bet a lot of money that they aren't actually breaking even on their API (or even close to). They don't have a "pay as you go" per-token tier, it's all geared up to demonstrate use of their API as a novelty. They're probably burning cash on every single token. But their valuation and hype has surely gone way up since they got onto LLMs.

They seem to have dev tier pricing (https://inference-docs.cerebras.ai/support/pricing). It's likely that they don't make much money on this and only make money on large enterprise contracts.

Re: Cerebras achieves 2,500T/s on Llama 4 Maverick (400B)

#59

Post by skryl »

ryao wrote: post #3

> At over 2,500 t/s, Cerebras has set a world record for LLM inference speed on the 400B parameter Llama 4 Maverick model, the largest and most powerful in the Llama 4 family. This is incorrect. The unreleased Llama 4 Behemoth is the largest and most powerful in the Llama 4 family. As for the speed record, it seems important to keep it in context. That comparison is only for performance on 1 query, but it is well kno…

Performance per watt is better than h100 and b200, performance per watt per $ is worse than B200, and it does fp8 just fine

https://arxiv.org/pdf/2503.11698

Re: Cerebras achieves 2,500T/s on Llama 4 Maverick (400B)

#60

Post by moralestapia »

acchow wrote: post #24
moralestapia wrote: post #15

Earlier quoted context omitted.

>Their hardware does inference with FP16, so they need ~20 of their CSE-3 chips to run this model. Care to explain? I don't see it.

CSE-3 chip has 44GB, which can hold 22B parameters in FP16. 400B parameters would need 18 chips. Then you need a bit more ram for other stuff

That's on-chip SRAM, comparable to a GPU's L1 cache, of which it typically has megabytes.

CSE systems also come with off-chip memory, comparable to a GPU's memory, but usually in the TB range.

Post reply on HN