Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Groq CEO: 'We No Longer Sell Hardware'

eetimes.com

Post reply on HN
Print view View original

11–20 of 152 posts

Re: Groq CEO: 'We No Longer Sell Hardware'

#11

Post by BoorishBears »

brcmthrowaway wrote: post #7
BoorishBears wrote: post #2

Read: We're forcing someone's hand in acquiring us. Groq is still under a 30 request per minute rate-limit, which drops to 10 requests per minute if you have all day usage. Billing has been "coming soon" this whole time, and while they've built out hype enabling features like function calling, somehow they can't setup a Stripe webhook to collect money for realistic rate limits. They couldn't scream "we can't service…

What are you using all this for? Whats the product?

I run an AI story telling site and an AI ideation platform.

The story telling site alone averaged 27k requests a day this week, so about double what their current request limit is, and honestly not even that popular of a site.

You can't run much more than a toy project on their current rate limits.

Re: Groq CEO: 'We No Longer Sell Hardware'

#12

Post by jsheard »

scosman wrote: post #3

Wildly fast inference. And current chips are 14nm so headroom to get a lot better.

Note that SRAM density doesn't scale at the same rate as logic density, and Groqs "secret sauce" is putting a ton of SRAM on their chips. Their stuff won't necessarily see the full benefits of switching to denser nodes if the bottleneck is how much SRAM they can pack onto each chip.

https://www.tomshardware.com/news/no-sram-scaling-implies-on...

IIRC the last big jump for SRAM density was at 7nm, so they do still have that card to play, but progress has slowed to a crawl beyond that. TSMC 3nm SRAM is barely denser than TSMC 7nm SRAM.

Re: Groq CEO: 'We No Longer Sell Hardware'

#13

Post by zetazzed »

Man, I want to appreciate a nice new hardware approach, but they say such BS that it is hard to read about them:

> “There might need to be a new term, because by the end of next year we’re going to deploy enough LPUs that compute-wise, it’s going to be the equivalent of all the hyperscalers combined,” he said. “We already have a non-trivial portion of that.”

Really? Does anyone seriously believe they are going to be the equivalent of all hyperscalers in compute next year? (Where Meta alone is at 1 million H100 equivalents.) In the same article where they say it's too hard for them to sell chips? And when they literally don't have a setup to even accept a credit card today?

Re: Groq CEO: 'We No Longer Sell Hardware'

#14

Post by latchkey »

Interesting, I guess that is why I never got a response back from them about buying their stuff.

My guess is that they realized that just selling hardware is a lot harder than running it themselves. Deploying this level of compute is non-trivial, with very high rates of failure, as well as huge supply chain issues. If you have to sell the hardware and support people buying it, that is a world of trouble.

> no-one wants to take the risk of buying a whole bunch of hardware

I do!

Nobody has stated it yet, but this is probably great news for tenstorrent.

Disclosure: building a cloud compute provider starting with AMD MI300x, and eventually any other high end hardware that our customers are asking for.

Re: Groq CEO: 'We No Longer Sell Hardware'

#15

Post by creato »

> If customers come with requests for high volumes of chips for very large installations, Groq will instead propose partnering on data center deployment. Ross said that Groq has “signed a deal” with Saudi state-owned oil company Aramco, though he declined to give further details, saying only that the deal involved “a very large deployment of [Groq] LPUs.”

What? How does this make sense?

Re: Groq CEO: 'We No Longer Sell Hardware'

#16

Post by rnts08 »

latchkey wrote: post #14

Interesting, I guess that is why I never got a response back from them about buying their stuff. My guess is that they realized that just selling hardware is a lot harder than running it themselves. Deploying this level of compute is non-trivial, with very high rates of failure, as well as huge supply chain issues. If you have to sell the hardware and support people buying it, that is a world of trouble. > no-one wan…

That's interesting, something that I've been really wanting to get into as well, but where I am there is literally no venture capital to raise for this at the moment. I'd be interested to know more and/or bounce some ideas though.

Re: Groq CEO: 'We No Longer Sell Hardware'

#17

Post by theturtletalks »

creato wrote: post #15

> If customers come with requests for high volumes of chips for very large installations, Groq will instead propose partnering on data center deployment. Ross said that Groq has “signed a deal” with Saudi state-owned oil company Aramco, though he declined to give further details, saying only that the deal involved “a very large deployment of [Groq] LPUs.” What? How does this make sense?

If you read on, Groq said they would only sell hardware to US companies and outside companies would get cloud services, not the LPUs. I think the US government told them to keep the LPUs in-house since they could be the secret sauce for scale.

Re: Groq CEO: 'We No Longer Sell Hardware'

#18

Post by rnts08 »

Sounds like they're looking to get bought up to me. I'm sure they could monetize their current hardware, and build to sell just like other niche hardware vendors. Anyone remember the hype around big "cloud" storage boxes 10 years back?

Re: Groq CEO: 'We No Longer Sell Hardware'

#19

Post by ipsum2 »

gandalfgeek wrote: post #6

They're calling the lie on needing bleeding edge hardware for performance. 5 yr old silicon (14 nm!!) and no hbm. Their secret sauce seems to be an ahead-of-time compiler that statically lays out entire computation, enabling zero contention at runtime. Basically, they stamp out all non-determinism. https://wow.groq.com/isca-2022-paper

It's not really a lie though. They require 20x more chips (storing all the weights in sram instead of hbm is expensive!) than Nvidia GPUs for a ~2x speed increase. Overall the power cost is more expensive for groq than GPUs.
Post reply on HN