Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B available on Cerebras at 1500 tokens/s

inference-docs.cerebras.ai

Post reply on HN
Print view View original

51–60 of 238 posts

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#51

Post by srcreigh »

tacone wrote: post #14

Noticed they are present in OpenRouter, but Qwen 3.8 is not there yet. Hopefully it'll get there soon. For those who haven't noticed though, the context size they allow for Qwen is just 128k. Still interesting as a specialized sub-agent but not really well suited for long tasks.

Great observation. That’s not enough context even for some one shot xhigh requests.

When I put Qwen3.8 27B xhigh towards adding scope proxying to the Guice library, it one shotted a great impl using 250k context before stopping.

Part of the greatness of the model is that it just keeps going until it gets a great result. 128k context is disappointing.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#52

Post by peri-cl »

hexa00 wrote: post #50

Just tried it on a medium size coding/debug problem on an existing codebase, observations: - Input doesn't look faster than other models, it spends a lot of time reading Read about 5M tokens - Output is awesome, super fast as you expect from the 1500t/sec I think that's correct - Tool call is failing more than say DS4, which leads to time wasted on retries (complex tools like browser control for example) - Shell comm…

> "Also maybe my setup (OMP) doesn't do the cache correctly but that's a huge cost driver... so atm it's quite pricy"

I don't believe Cerebras has a cached input pricing? They don't list one on the model page:

https://inference-docs.cerebras.ai/models/qwen-3.8-27b

edit: See the sibling discussion,

https://news.ycombinator.com/item?id=49554520#49555094 ("Input tokens, whether served from the cache or processed fresh, are billed at the standard input token rate")

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#53

Post by bitexploder »

freehorse wrote: post #33

I have used their gemma 4 31b model through kagi and getting real instantaneous answers is absolutely crazy. A very different feeling and UX. Even if the model is smaller, there is definitely a use case for these. I was wondering if they would put the qwen 27b model, it sounds very interesting to try.

The thing I didn’t realize for a while is 27B is rather smart. As many (or more) activated parameters as the flash models of the universe that we know about. It reasons very well. It just doesn’t have a lot of knowledge.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#54

Post by nostrebored »

150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool.

Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information.

``` Billing access restricted Self-serve billing is not available on Enterprise accounts. Please contact your team for further questions. ```

We have no team (they removed themself from our slack channel after we talked about rate limits). Perplexingly, none of this even shows up in the request, which gives:

``` {"message":"Model does not exist or you do not have access to it.","type":"not_found_error","param":"model","code":"model_not_found"} ```

When the error is really about billing.

I always want to like Cerebras, but I get the vibe that as a tokens in tokens out consumer you are not valued at all.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#55

Post by danielklnstein »

zackangelo wrote: post #31
jasongill wrote: post #16

It would be great if they made their inference capacity for this model available via OpenRouter; the fastest provider on OpenRouter right now is at ~80tps https://openrouter.ai/qwen/qwen3.8-27b#providers They do appear to host other models on OpenRouter so maybe Qwen3.8 will be there soon: https://openrouter.ai/provider/cerebras

We're serving it around 150-200tok/s (uses our new speculative decoding implementation on a DFlash2 draft model). https://mixlayer.com , LAUNCH-Q38-27B gets you $5 in credits if you want to kick the tires.

I tried in your playground and got 14.2 tok/s?

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#56

Post by beastman82 »

pllbnk wrote: post #48

Just a couple days ago I learned about ninfer ( https://github.com/Neroued/ninfer ) and on RTX 5090 I can now get ~200 tok/s and over 400 tok/s on concurrent requests which is plenty fast for a local model of this strength.

can't second ninfer enough. amazing tech

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#58

Post by zackangelo »

danielklnstein wrote: post #55
zackangelo wrote: post #31

Earlier quoted context omitted.

We're serving it around 150-200tok/s (uses our new speculative decoding implementation on a DFlash2 draft model). https://mixlayer.com , LAUNCH-Q38-27B gets you $5 in credits if you want to kick the tires.

I tried in your playground and got 14.2 tok/s?

apologies we just got a sudden burst of new users and traffic, it's scaling up now.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#59

Post by minimaltom »

codexon wrote: post #17
porphyra wrote: post #4

Why do they only host small models rather than the 2.4T version? Is the I/O and interconnect between the wafers bad due to the limited beachfront relative to the massive size of the chip?

The wafer only has space for 44 gb of sram. If they offload ram they lose the speedup of having everything on 1 chip (the whole point of cerebras).

[dead]

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#60

Post by srcreigh »

the_duke wrote: post #36

Funnily enough the pricing isn't that much worse than on openrouter, where the best price at the moment is $0.24 in / $2.55 out, vs $1 / $1.5 on Cerebras. Sure, 4x input , but cheaper output. Though Cerebras doesn't have prompt caching, so not great for agentic workloads. (they do, but it doesn't affect the price.

It is 15x more expensive. Openrouter usually charges like 1/4 for cached input.

Most of the cost for agentic coding is input tokens, you pay for the whole context at each tool call or message. Output tokens is just a small rate

Post reply on HN