Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B available on Cerebras at 1500 tokens/s

inference-docs.cerebras.ai

Post reply on HN
Print view View original

111–120 of 240 posts

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#111

Post by codazoda »

low_tech_punk wrote: post #107
codazoda wrote: post #106

Do I understand their pricing correctly? This is $10 per month for a developer account PLUS you pay $1.49/M for output tokens and $0.99/M for input tokens on Qwen 3.8 27b with a 128k context? EDIT: Or, maybe it's just token pricing, but $10 is the minimum? Maybe it's that. https://www.cerebras.ai/pricing

No. You buy a minimum of $10 worth of credit, then use it at $1.49/M rate. There is no recurring charge. There is a separate subscription based plan, which is sold out now.

Got it. But, they also charge the same for cached tokens, so that probably closes the gap on Foundation models quite a bit.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#112

Post by codazoda »

freehorse wrote: post #33

I have used their gemma 4 31b model through kagi and getting real instantaneous answers is absolutely crazy. A very different feeling and UX. Even if the model is smaller, there is definitely a use case for these. I was wondering if they would put the qwen 27b model, it sounds very interesting to try.

Really an aside, but yesterday I got the Gemma-4-12b (128k context) to build it's first web app in the minimal Dark Software Factory I've been building for myself.

https://joeldare.com/a-local-open-weight-model-builds-its-fi...

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#113

Post by Aurornis »

nostrebored wrote: post #54

150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool. Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information. ``` Billing access restricted Self-serve billi…

> 150k TPM limit on public endpoint means that it's likely unusable for many coding tasks.

I don't understand. How does that make it unusable? Is the limit shared by an entire team at once?

150,000 tokens per minute is a lot. You could start hitting that with a lot of concurrent requests in your session, but even throttled to 150k TPM it's still going to be faster than anything else you find.

I think the 128K context limit is the real ceiling. These models aren't amazing at long context, but once you account for a short input prompt, the input files, and headroom for a compaction summary, there isn't a lot left for the problem.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#114

Post by bitexploder »

nicce wrote: post #100
bitexploder wrote: post #53

Earlier quoted context omitted.

The thing I didn’t realize for a while is 27B is rather smart. As many (or more) activated parameters as the flash models of the universe that we know about. It reasons very well. It just doesn’t have a lot of knowledge.

They seem to have good enough general intelligence that missing knowledge is not that big thing. If you are able to have a proper [free search engine], they can do almost anything. Having own local search index about relevant stuff can help a lof if you don’t want to pay for search API.

But running that fast… with a local RAG? Yeah, it is a very interesting model. Maybe you don’t need a lot of parameters, just a really big local database :)

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#115

Post by codazoda »

ecshafer wrote: post #83

I have a self hosted Qwen 3.8 27B and I find it to be unusably bad. Using it agentically, it will spin around in circles on even small tasks talking to itself until it loses context and starts again. I even had it say "I've forgotten the users initial question"

I want a Qwen 3.8 27B hosted locally but I don't quite have the RAM for it. And, I don't want to buy the RAM until I prove I can use it.

Yesterday I did have success with Gemma-4-12b with 128k context. It fits in my RAM and it's relatively fast on my hardware.

I had to give it prompts that are quite a bit different from the way I use foundation models, but I did get it to work quite well. I feel like I could learn it's differences and get good at using it for real work.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#116

Post by conception »

Aurornis wrote: post #113
nostrebored wrote: post #54

150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool. Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information. ``` Billing access restricted Self-serve billi…

> 150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. I don't understand. How does that make it unusable? Is the limit shared by an entire team at once? 150,000 tokens per minute is a lot. You could start hitting that with a lot of concurrent requests in your session, but even throttled to 150k TPM it's still going to be faster than anything else you find. I think the 128K context…

150k by account. At 1.5k a second you hit it very quickly.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#117

Post by RussianCow »

fc417fc802 wrote: post #108
baegi wrote: post #95

Earlier quoted context omitted.

Not sure how the rate limiting works, but it's 1.5k TPS, not 15k, so you could run it for 100s/min, which seems good enough to me

It seems you forgot to account for the fact that cerebras uses a baker's minute which is 144 seconds instead of 60. (Seriously though what's the supposed issue here?)

The issue is that all input (including context) counts towards that limit. So 10 requests with 50k of context will blow through the limit, even if little to no output was generated, which is incredibly easy to do with agentic workloads.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#118

Post by RussianCow »

zackangelo wrote: post #31
jasongill wrote: post #16

It would be great if they made their inference capacity for this model available via OpenRouter; the fastest provider on OpenRouter right now is at ~80tps https://openrouter.ai/qwen/qwen3.8-27b#providers They do appear to host other models on OpenRouter so maybe Qwen3.8 will be there soon: https://openrouter.ai/provider/cerebras

We're serving it around 150-200tok/s (uses our new speculative decoding implementation on a DFlash2 draft model). https://mixlayer.com , LAUNCH-Q38-27B gets you $5 in credits if you want to kick the tires.

I don't see any kind of input cache discount listed on your pricing page. Do you offer that, or is all input priced the same?

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#119

Post by devy »

conception wrote: post #116
Aurornis wrote: post #113

Earlier quoted context omitted.

> 150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. I don't understand. How does that make it unusable? Is the limit shared by an entire team at once? 150,000 tokens per minute is a lot. You could start hitting that with a lot of concurrent requests in your session, but even throttled to 150k TPM it's still going to be faster than anything else you find. I think the 128K context…

150k by account. At 1.5k a second you hit it very quickly.

Exactly, it burns the tokens 3000x faster, which means the budget ($$$$$$) runs out so faster it will stop super quick, not able to perform long-duration work. At 27B parameter size, the intelligence is not able to accomplish work within a short amount time. Consequently, it become not usable.

Re: Qwen 3.8 27B available on Cerebras at 1500 tokens/s

#120

Post by datadrivenangel »

Aurornis wrote: post #113
nostrebored wrote: post #54

150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool. Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information. ``` Billing access restricted Self-serve billi…

> 150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. I don't understand. How does that make it unusable? Is the limit shared by an entire team at once? 150,000 tokens per minute is a lot. You could start hitting that with a lot of concurrent requests in your session, but even throttled to 150k TPM it's still going to be faster than anything else you find. I think the 128K context…

150k tokens per minute at 1.5k tokens per second means you can have like 3 users concurrently and that's not a lot.
Post reply on HN