Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B available on Cerebras at 1500 tokens/s

inference-docs.cerebras.ai

Post reply on HN
Print view View original

61–70 of 240 posts

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#61

Post by gpugreg »

I was wondering whether this was any good for programming, but it is too fast for its own good. There is a limit of 450,000 tokens per minute. I hit this limit in about 90 seconds and burned through $1.10 while doing so. This is because cached tokens count towards the token limit.

For comparison, I ran the same task with DeepSeek-V4-Flash, which finished in 172 seconds and cost $0.024 with a final context window size of 55217 tokens, while Qwen3.8-27B was not even close to being done with a 64178 context window.

This is a very efficient way to burn your money, but I would not recommend it for programming.

On the positive side, I got a $5 signup bonus, so it wasn't my own money.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#62

Post by olivermuty »

nostrebored wrote: post #54

150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool. Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information. ``` Billing access restricted Self-serve billi…

Cerebras the tech is awesome, cerebras the company is a trainwreck

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#63

Post by Zambyte »

londons_explore wrote: post #24
foundfontic wrote: post #8

I really wish they had their customer support somewhere else than Discord, which seems to think I'm a bot and doesen't accept my email or phone numbe

discord support can fix such issues

Discord is simply a liability.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#64

Post by olivermuty »

peri-cl wrote: post #52
hexa00 wrote: post #50

Just tried it on a medium size coding/debug problem on an existing codebase, observations: - Input doesn't look faster than other models, it spends a lot of time reading Read about 5M tokens - Output is awesome, super fast as you expect from the 1500t/sec I think that's correct - Tool call is failing more than say DS4, which leads to time wasted on retries (complex tools like browser control for example) - Shell comm…

> "Also maybe my setup (OMP) doesn't do the cache correctly but that's a huge cost driver... so atm it's quite pricy" I don't believe Cerebras has a cached input pricing? They don't list one on the model page: https://inference-docs.cerebras.ai/models/qwen-3.8-27b edit: See the sibling discussion, https://news.ycombinator.com/item?id=49554520#49555094 ( "Input tokens, whether served from the cache or processed fresh,…

They have cache, but it costs the same indeed, no idea what the point of the cache is

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#65

Post by vb-8448 »

yipinwong wrote: post #44
vb-8448 wrote: post #23

At that speed it's too pricey for agentinc tasks.

The target audience is who needs raw speed. Having the choice is good as you can make a trade-off between speed, perf, and quality. Until last year, people had a single AI god they believed in (mostly Anthropic stuff). Now we have power to make choices (open-weights, SOTA, speed-optimized, etc) the same way you do for system designs.

It's not a criticism, I was really looking forward to trying out such a powerful model at this speed.

But I burn my 5$ allowance in 10 minutes ... and only because I was hitting rate limits, without it would probably be less than a minute.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#66

Post by lostmsu »

olivermuty wrote: post #64
peri-cl wrote: post #52

Earlier quoted context omitted.

> "Also maybe my setup (OMP) doesn't do the cache correctly but that's a huge cost driver... so atm it's quite pricy" I don't believe Cerebras has a cached input pricing? They don't list one on the model page: https://inference-docs.cerebras.ai/models/qwen-3.8-27b edit: See the sibling discussion, https://news.ycombinator.com/item?id=49554520#49555094 ( "Input tokens, whether served from the cache or processed fresh,…

They have cache, but it costs the same indeed, no idea what the point of the cache is

They don't have cache (e.g. KV cache). But they write down what you sent earlier to say they cached it! To still bill the same as uncached later (because they didn't actually cache it)!

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#68

Post by lostmsu »

Barbing wrote: post #43
abtinf wrote: post #35

Earlier quoted context omitted.

> How are cached tokens priced? > There is no additional fee for using prompt caching. Input tokens, whether served from the cache or processed fresh, are billed at the standard input token rate for the respective model. Well, talk about flipping the narrative.

heh Is there a speed increase or is that purely marketing spin on “we might cache on our end but no discount for you”?

Pure marketing.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#70

Post by d2p »

gpugreg wrote: post #61

I was wondering whether this was any good for programming, but it is too fast for its own good. There is a limit of 450,000 tokens per minute. I hit this limit in about 90 seconds and burned through $1.10 while doing so. This is because cached tokens count towards the token limit. For comparison, I ran the same task with DeepSeek-V4-Flash, which finished in 172 seconds and cost $0.024 with a final context window size…

> There is a limit of 450,000 tokens per minute. I hit this limit in about 90 seconds

I'm confused. If it's 1500t/s, isn't that only 90k per minute? How do you hit a 450k/minute limit?

Post reply on HN