Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Groq CEO: 'We No Longer Sell Hardware'

eetimes.com

Post reply on HN
Print view View original

21–30 of 152 posts

Re: Groq CEO: 'We No Longer Sell Hardware'

#21

Post by htrp »

latchkey wrote: post #14

Interesting, I guess that is why I never got a response back from them about buying their stuff. My guess is that they realized that just selling hardware is a lot harder than running it themselves. Deploying this level of compute is non-trivial, with very high rates of failure, as well as huge supply chain issues. If you have to sell the hardware and support people buying it, that is a world of trouble. > no-one wan…

that or if you put chips in the hands of your customers, they may start to benchmark it against other equivalent solutions

Re: Groq CEO: 'We No Longer Sell Hardware'

#22

Post by creato »

theturtletalks wrote: post #17
creato wrote: post #15

> If customers come with requests for high volumes of chips for very large installations, Groq will instead propose partnering on data center deployment. Ross said that Groq has “signed a deal” with Saudi state-owned oil company Aramco, though he declined to give further details, saying only that the deal involved “a very large deployment of [Groq] LPUs.” What? How does this make sense?

If you read on, Groq said they would only sell hardware to US companies and outside companies would get cloud services, not the LPUs. I think the US government told them to keep the LPUs in-house since they could be the secret sauce for scale.

I'm not questioning the deployment strategy, I'm wondering why Saudi Aramco wants to access so much compute power that is highly specialized(?) for generative AI workloads. Or is it more general than that?

Re: Groq CEO: 'We No Longer Sell Hardware'

#23

Post by htrp »

Havoc wrote: post #20

Given that their hardware is different I can kinda see how they don’t want to deal with supporting customers. > what do you mean I can’t just drop a CUDA docker image in?

if you're a hardware startup that doesn't sell hardware, what are you?

Re: Groq CEO: 'We No Longer Sell Hardware'

#25

Post by germanjoey »

halflings wrote: post #8
gandalfgeek wrote: post #6

They're calling the lie on needing bleeding edge hardware for performance. 5 yr old silicon (14 nm!!) and no hbm. Their secret sauce seems to be an ahead-of-time compiler that statically lays out entire computation, enabling zero contention at runtime. Basically, they stamp out all non-determinism. https://wow.groq.com/isca-2022-paper

No HBM because they use tons of fast SRAM instead. Isn't that the main driver for performance here? (the way I understood it => it's still cost effective at scale due to throughput increase this brings)

cost effective in what sense? groq doesn't achieve high efficiency, only low latency. but that's not done in a cost-effective way. compare sambanova achieving the same performance with 8 chips instead of 568, and with higher precision.

Re: Groq CEO: 'We No Longer Sell Hardware'

#26

Post by latchkey »

htrp wrote: post #21
latchkey wrote: post #14

Interesting, I guess that is why I never got a response back from them about buying their stuff. My guess is that they realized that just selling hardware is a lot harder than running it themselves. Deploying this level of compute is non-trivial, with very high rates of failure, as well as huge supply chain issues. If you have to sell the hardware and support people buying it, that is a world of trouble. > no-one wan…

that or if you put chips in the hands of your customers, they may start to benchmark it against other equivalent solutions

Funny you should mention that. ;-)

https://www.reddit.com/r/LocalLLaMA/comments/1bpgrdf/wanted_...

I've got about a dozen people signed up. Just working through some hardware issues right now (see above about high rate of failures), and hope to have this resolved next week, so that I can get people onto them and doing their testing.

Re: Groq CEO: 'We No Longer Sell Hardware'

#27

Post by aleph_minus_one »

latchkey wrote: post #14

Interesting, I guess that is why I never got a response back from them about buying their stuff. My guess is that they realized that just selling hardware is a lot harder than running it themselves. Deploying this level of compute is non-trivial, with very high rates of failure, as well as huge supply chain issues. If you have to sell the hardware and support people buying it, that is a world of trouble. > no-one wan…

> If you have to sell the hardware and support people buying it, that is a world of trouble.

What is the difference between this and having to sell the cloud access and supporting the people who buy a subscription?

Re: Groq CEO: 'We No Longer Sell Hardware'

#28

Post by LoganDark »

dsrtslnd23 wrote: post #9

So unless there are new Croq datacenters coming, this is only interesting for North American users. Otherwise H100 based latency optimized solutions would be faster - in particular for time-to-first-token sensitive applications.

> latency optimized solutions would be faster - in particular for time-to-first-token sensitive applications

Do you have any idea how fast Groq is? Go try it. Consistently over 400 t/s for most of the models that they support, and extremely low latency.

Re: Groq CEO: 'We No Longer Sell Hardware'

#29

Post by aleph_minus_one »

htrp wrote: post #23
Havoc wrote: post #20

Given that their hardware is different I can kinda see how they don’t want to deal with supporting customers. > what do you mean I can’t just drop a CUDA docker image in?

if you're a hardware startup that doesn't sell hardware, what are you?

> if you're a hardware startup that doesn't sell hardware, what are you?

A hardware startup that sells cloud access to its hardware. :-)

Re: Groq CEO: 'We No Longer Sell Hardware'

#30

Post by latchkey »

rnts08 wrote: post #16
latchkey wrote: post #14

Interesting, I guess that is why I never got a response back from them about buying their stuff. My guess is that they realized that just selling hardware is a lot harder than running it themselves. Deploying this level of compute is non-trivial, with very high rates of failure, as well as huge supply chain issues. If you have to sell the hardware and support people buying it, that is a world of trouble. > no-one wan…

That's interesting, something that I've been really wanting to get into as well, but where I am there is literally no venture capital to raise for this at the moment. I'd be interested to know more and/or bounce some ideas though.

Extremely capital intensive, but also requires relationships in the industry at many levels. Luckily, I happen to have both and they are crazy enough to put their trust in me. I feel very grateful for that.
Post reply on HN