Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B available on Cerebras at 1500 tokens/s

inference-docs.cerebras.ai

Post reply on HN
Print view View original

81–90 of 239 posts

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#81

Post by eli »

irthomasthomas wrote: post #75
eli wrote: post #74

I just did a little anecdotal test. Had pi + cerebras review a recent commit and asked a few quick followups on it. Worked great. The Cerebras session cost me $1.60 and took a total of 5.1 mins. I did get a few brief 429 rate limit errors in there. The p50 speed was 890 tok/s and 0.64s TTFT. Using OpenRouter averages, that would've cost $0.29 (no cache discount at Cerebras!) and would've taken about 14.4 minutes. So…

Thanks! Is there something about their platform that prevents caching? Or are they just not passing on the discount?

The session had a 91.4% cache hit rate. They just give zero discount.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#83

Post by ecshafer »

I have a self hosted Qwen 3.8 27B and I find it to be unusably bad. Using it agentically, it will spin around in circles on even small tasks talking to itself until it loses context and starts again. I even had it say "I've forgotten the users initial question"

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#84

Post by FeepingCreature »

ecshafer wrote: post #83

I have a self hosted Qwen 3.8 27B and I find it to be unusably bad. Using it agentically, it will spin around in circles on even small tasks talking to itself until it loses context and starts again. I even had it say "I've forgotten the users initial question"

I have a self hosted Qwen 3.8 27B and I find it unbelievably cracked and dedicated. It's at least credibly attempted everything I've thrown at it. Just today I had it write a toy compiler with a JIT backend just to test out a concept, and that was with 4-bit quantization and 8-bit KV cache. Something has to be going wrong with your deployment.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#86

Post by redman25 »

irthomasthomas wrote: post #80
hexa00 wrote: post #50

Just tried it on a medium size coding/debug problem on an existing codebase, observations: - Input doesn't look faster than other models, it spends a lot of time reading Read about 5M tokens - Output is awesome, super fast as you expect from the 1500t/sec I think that's correct - Tool call is failing more than say DS4, which leads to time wasted on retries (complex tools like browser control for example) - Shell comm…

I can't believe this situation has not improved in years. Is cerebras' main business selling the hardware, then?

Maybe they’re gunning for speedy non-interactive pricing? Or its a limit of the technology or a business decision?

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#87

Post by zackangelo »

danielklnstein wrote: post #55
zackangelo wrote: post #31

Earlier quoted context omitted.

We're serving it around 150-200tok/s (uses our new speculative decoding implementation on a DFlash2 draft model). https://mixlayer.com , LAUNCH-Q38-27B gets you $5 in credits if you want to kick the tires.

I tried in your playground and got 14.2 tok/s?

just added 8 more H200s to the cluster, if you (or anyone else) runs into issues please feel free to drop me a message: zack at mixlayer.com

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#88

Post by Pxtl »

gpugreg wrote: post #61

I was wondering whether this was any good for programming, but it is too fast for its own good. There is a limit of 450,000 tokens per minute. I hit this limit in about 90 seconds and burned through $1.10 while doing so. This is because cached tokens count towards the token limit. For comparison, I ran the same task with DeepSeek-V4-Flash, which finished in 172 seconds and cost $0.024 with a final context window size…

Could this also be coming from the problem that Qwen3.8-27B's default mode being "extra-high reasoning level"?

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#89

Post by [deleted] »

nostrebored wrote: post #54

150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool. Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information. ``` Billing access restricted Self-serve billi…

[deleted]

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#90

Post by 0xbadcafebee »

nostrebored wrote: post #54

150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool. Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information. ``` Billing access restricted Self-serve billi…

Yeah, their public service isn't a serious/competitive offering. They don't have the capacity to serve all the customers who might want to use them at that speed. The public service exists so they get some users on OpenRouter, and that shows them as #1 on speed, which proves their tech is very fast, which gets them billions in hardware sales/licensing. If you have big enough pockets they can probably dedicate capacity to you. But for reliably fast small models you might want to rent some GPUs.
Post reply on HN