Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B available on Cerebras at 1500 tokens/s

inference-docs.cerebras.ai

Post reply on HN
Print view View original

71–80 of 238 posts

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#71

Post by hexa00 »

peri-cl wrote: post #52
hexa00 wrote: post #50

Just tried it on a medium size coding/debug problem on an existing codebase, observations: - Input doesn't look faster than other models, it spends a lot of time reading Read about 5M tokens - Output is awesome, super fast as you expect from the 1500t/sec I think that's correct - Tool call is failing more than say DS4, which leads to time wasted on retries (complex tools like browser control for example) - Shell comm…

> "Also maybe my setup (OMP) doesn't do the cache correctly but that's a huge cost driver... so atm it's quite pricy" I don't believe Cerebras has a cached input pricing? They don't list one on the model page: https://inference-docs.cerebras.ai/models/qwen-3.8-27b edit: See the sibling discussion, https://news.ycombinator.com/item?id=49554520#49555094 ( "Input tokens, whether served from the cache or processed fresh,…

lol yeah just saw that, yeah that makes it unusable I think at least for me.

I wonder if they will do that with sol ultrafast!

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#72

Post by bookernath »

zackangelo wrote: post #31
jasongill wrote: post #16

It would be great if they made their inference capacity for this model available via OpenRouter; the fastest provider on OpenRouter right now is at ~80tps https://openrouter.ai/qwen/qwen3.8-27b#providers They do appear to host other models on OpenRouter so maybe Qwen3.8 will be there soon: https://openrouter.ai/provider/cerebras

We're serving it around 150-200tok/s (uses our new speculative decoding implementation on a DFlash2 draft model). https://mixlayer.com , LAUNCH-Q38-27B gets you $5 in credits if you want to kick the tires.

This feels great

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#73

Post by nkhs89 »

hexa00 wrote: post #50

Just tried it on a medium size coding/debug problem on an existing codebase, observations: - Input doesn't look faster than other models, it spends a lot of time reading Read about 5M tokens - Output is awesome, super fast as you expect from the 1500t/sec I think that's correct - Tool call is failing more than say DS4, which leads to time wasted on retries (complex tools like browser control for example) - Shell comm…

[dead]

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#74

Post by eli »

I just did a little anecdotal test. Had pi + cerebras review a recent commit and asked a few quick followups on it. Worked great.

The Cerebras session cost me $1.60 and took a total of 5.1 mins. I did get a few brief 429 rate limit errors in there. The p50 speed was 890 tok/s and 0.64s TTFT.

Using OpenRouter averages, that would've cost $0.29 (no cache discount at Cerebras!) and would've taken about 14.4 minutes.

So on this one short session, cerebras was 5.6x more expensive in exchange for being 2.8x faster. Or, another way, $1.32 buys back about 9 minutes of your time. Not a bad trade IMHO but the cache situation is a real bummer. The longer your session the more relatively expensive Cerebras gets. The "good" news is you're also limited by its short context window.

(Also, I used to be on the Cerebras coding plan and the support is pretty bad for end users. My guess is these public endpoints are really just product demos for potential enterprise customers.)

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#75

Post by irthomasthomas »

eli wrote: post #74

I just did a little anecdotal test. Had pi + cerebras review a recent commit and asked a few quick followups on it. Worked great. The Cerebras session cost me $1.60 and took a total of 5.1 mins. I did get a few brief 429 rate limit errors in there. The p50 speed was 890 tok/s and 0.64s TTFT. Using OpenRouter averages, that would've cost $0.29 (no cache discount at Cerebras!) and would've taken about 14.4 minutes. So…

Thanks! Is there something about their platform that prevents caching? Or are they just not passing on the discount?

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#76

Post by gpugreg »

d2p wrote: post #70
gpugreg wrote: post #61

I was wondering whether this was any good for programming, but it is too fast for its own good. There is a limit of 450,000 tokens per minute. I hit this limit in about 90 seconds and burned through $1.10 while doing so. This is because cached tokens count towards the token limit. For comparison, I ran the same task with DeepSeek-V4-Flash, which finished in 172 seconds and cost $0.024 with a final context window size…

> There is a limit of 450,000 tokens per minute. I hit this limit in about 90 seconds I'm confused. If it's 1500t/s, isn't that only 90k per minute? How do you hit a 450k/minute limit?

Cached tokens count towards the limit as well. For example, if your context window is 50,000 tokens, it takes 9 requests to reach that limit without generating a single token.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#77

Post by orliesaurus »

Qwen 3.8 27B is an exceptional model for coding and ranks as one of the best local models for coding....BUT in my head I am confused why a company that's IPO'd doesn't invest in RL'd super specialized, super-damn-fast models for very specific tasks - instead of giving us the OSS GPT model from what feels like 200 years ago

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#80

Post by irthomasthomas »

hexa00 wrote: post #50

Just tried it on a medium size coding/debug problem on an existing codebase, observations: - Input doesn't look faster than other models, it spends a lot of time reading Read about 5M tokens - Output is awesome, super fast as you expect from the 1500t/sec I think that's correct - Tool call is failing more than say DS4, which leads to time wasted on retries (complex tools like browser control for example) - Shell comm…

I can't believe this situation has not improved in years. Is cerebras' main business selling the hardware, then?
Post reply on HN