Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-OSS 120B Runs at 3000 tokens/sec on Cerebras

cerebras.ai

Post reply on HN
Print view View original

11–20 of 31 posts

Re: GPT-OSS 120B Runs at 3000 tokens/sec on Cerebras

#11

Post by freak42 »

Saline9515 wrote: post #10
traceroute66 wrote: post #7

Earlier quoted context omitted.

Headline at the top of the Cerebras page linked to by the OP "Cerebras Raises $1.1B Series G at $8.1B Valuation" . If you're going after the AI money gravy train then you need to wave the "we have $n registered users" carrot on your PPT slides for the investors because registered user == monetization opportunity . I'm not defending it. I hate being forced to register for shit when I just want to try it or use the fre…

Well if they give it out for free (aka they pay for it), asking you to register is a reasonable ask. It's not a public service funded by taxpayers.

Yes they can ask, but do it at the beginning not the end of the process, this is a dark pattern and fucking annoying.

Re: GPT-OSS 120B Runs at 3000 tokens/sec on Cerebras

#12

Post by petesergeant »

jsheard wrote: post #8
petesergeant wrote: post #5

It’s an absolute beast. I run it via OpenRouter, where I have Groq and Cerebras as the providers. Cheap enough as to be almost free, strong performance, and lightning fast.

Cheap enough for now, but of all the companies selling inference at a loss, Cerebras and Groq are probably losing the most per token. Their hardware is ungodly expensive and its reliance on huge amounts of SRAM bottlenecks how much cheaper it can get, since SRAM density is improving at a snails pace at this point.

Not doubting you but anything to back that up? Happy enough to burn VC money until someone shows up who can run it without losing money, either way.

Re: GPT-OSS 120B Runs at 3000 tokens/sec on Cerebras

#13

Post by magackame »

freak42 wrote: post #11
Saline9515 wrote: post #10

Earlier quoted context omitted.

Well if they give it out for free (aka they pay for it), asking you to register is a reasonable ask. It's not a public service funded by taxpayers.

Yes they can ask, but do it at the beginning not the end of the process, this is a dark pattern and fucking annoying.

Anyone remember those online psychological tests where you spend an hour on one and in the end you need to pay up to get the result?)))

Re: GPT-OSS 120B Runs at 3000 tokens/sec on Cerebras

#14

Post by moralestapia »

freak42 wrote: post #2

I absolutely hate it, when a website says "try this" and after you went through the trouble of weiting something comes up with a sign up link first. Makes me leave instantly to never come back.

Off topic but related.

A week ago I went to a launch party for a product that's supposed to "revolutionize design" (a web app w/ an OAI prompt).

No demo, only like two pictures of the actual product. Founder spent like half an hour giving a speech about the future, etc...

"All of you here will get access to it in a couple weeks."

Couple weeks go by ... I "get access". It's a .dmg, 1) What, I open it, it's not even an app, it's an installer ..., I install it, the app opens up and it's a giant red button that takes you to a website to create an account ...

These guys are completely lost.

Re: GPT-OSS 120B Runs at 3000 tokens/sec on Cerebras

#16

Post by freak42 »

anonym29 wrote: post #9
freak42 wrote: post #2

I absolutely hate it, when a website says "try this" and after you went through the trouble of weiting something comes up with a sign up link first. Makes me leave instantly to never come back.

This is like declaring that a Ferrari dealership offering you a free test drive in a million dollar art exhibit on wheels is evil for asking for your phone number before handing you the keys. If this was some beat-to-hell, high-mileage used economy car, sure, that would be a pain in the ass, and not worth it. But it's a mistake to place Cerebras into that mental bucket. You don't even need to use real information to…

You didn't get my point at all.

Re: GPT-OSS 120B Runs at 3000 tokens/sec on Cerebras

#17

Post by traceroute66 »

Saline9515 wrote: post #10
traceroute66 wrote: post #7

Earlier quoted context omitted.

Headline at the top of the Cerebras page linked to by the OP "Cerebras Raises $1.1B Series G at $8.1B Valuation" . If you're going after the AI money gravy train then you need to wave the "we have $n registered users" carrot on your PPT slides for the investors because registered user == monetization opportunity . I'm not defending it. I hate being forced to register for shit when I just want to try it or use the fre…

Well if they give it out for free (aka they pay for it), asking you to register is a reasonable ask. It's not a public service funded by taxpayers.

> Well if they give it out for free (aka they pay for it), asking you to register is a reasonable ask

They have other options... rate limiting, serving (more) quantized to non-registered etc. etc.

Re: GPT-OSS 120B Runs at 3000 tokens/sec on Cerebras

#18

Post by rajman187 »

petesergeant wrote: post #12
jsheard wrote: post #8

Earlier quoted context omitted.

Cheap enough for now, but of all the companies selling inference at a loss, Cerebras and Groq are probably losing the most per token. Their hardware is ungodly expensive and its reliance on huge amounts of SRAM bottlenecks how much cheaper it can get, since SRAM density is improving at a snails pace at this point.

Not doubting you but anything to back that up? Happy enough to burn VC money until someone shows up who can run it without losing money, either way.

They’ve filed a S1 [1] last year when attempting to go public. It showed something like a $60M+ loss for the first 6 months of 2024. The IPO didn’t happen because the CEO’s past included some financial missteps and the banks didn’t want to deal with this. At the time the majority of their revenue came from a single source in Abu Dhabi, as well

[1] https://www.sec.gov/Archives/edgar/data/2021728/000162828024...

Re: GPT-OSS 120B Runs at 3000 tokens/sec on Cerebras

#19

Post by traceroute66 »

freak42 wrote: post #11
Saline9515 wrote: post #10

Earlier quoted context omitted.

Well if they give it out for free (aka they pay for it), asking you to register is a reasonable ask. It's not a public service funded by taxpayers.

Yes they can ask, but do it at the beginning not the end of the process, this is a dark pattern and fucking annoying.

> do it at the beginning not the end

Exactly this.

If you present me with a form and a submit button then I expect the input to go through and a result to be presented.

If you don't want to present me with results before login, then put the form behind the wall too.

Simple.

Re: GPT-OSS 120B Runs at 3000 tokens/sec on Cerebras

#20

Post by rpdillon »

freak42 wrote: post #16
anonym29 wrote: post #9

Earlier quoted context omitted.

This is like declaring that a Ferrari dealership offering you a free test drive in a million dollar art exhibit on wheels is evil for asking for your phone number before handing you the keys. If this was some beat-to-hell, high-mileage used economy car, sure, that would be a pain in the ass, and not worth it. But it's a mistake to place Cerebras into that mental bucket. You don't even need to use real information to…

You didn't get my point at all.

Would your expectations be more aligned if it's said "free trial"? That might create an expectation of a sign up where "try this" might not.
Post reply on HN