Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Groq CEO: 'We No Longer Sell Hardware'

eetimes.com

Post reply on HN
Print view View original

61–70 of 152 posts

Re: Groq CEO: 'We No Longer Sell Hardware'

#61

Post by htrp »

alephnerd wrote: post #38
latchkey wrote: post #37

Earlier quoted context omitted.

The pricing isn't necessarily much higher. The pricing is set up to amortize the cost of running things over a period of time. If you're only going to use it for a short time, you pay more than someone who commits to a 3 year contract. It also isn't just the hardware capex, it is everything involved under the covers. Market pricing also factors into that as well. This is something I've struggled with myself quite a b…

> it is everything involved under the covers Yep! You explained it better than me! > Groq probably realized that people buying 1-2 cards at a time, wasn't going to be profitable Yep! And if would have bogged them down by slowing down R&D cycles and even fulfilling orders as they obviously are not placing orders the same size as Nvidia or AMD. I wonder what this portends for SambaNova and other similar vendors as well…

>I wonder what this portends for SambaNova and other similar vendors as well.

They're focused on services based full stack deployments afaik.

They come in with a rack and sell you on models as well.

Re: Groq CEO: 'We No Longer Sell Hardware'

#62

Post by htrp »

latchkey wrote: post #26
htrp wrote: post #21

Earlier quoted context omitted.

that or if you put chips in the hands of your customers, they may start to benchmark it against other equivalent solutions

Funny you should mention that. ;-) https://www.reddit.com/r/LocalLLaMA/comments/1bpgrdf/wanted_... I've got about a dozen people signed up. Just working through some hardware issues right now (see above about high rate of failures), and hope to have this resolved next week, so that I can get people onto them and doing their testing.

Shiny! You have you guys gotten MI300X's working for non-inference use cases?

Re: Groq CEO: 'We No Longer Sell Hardware'

#63

Post by latchkey »

htrp wrote: post #62
latchkey wrote: post #26

Earlier quoted context omitted.

Funny you should mention that. ;-) https://www.reddit.com/r/LocalLLaMA/comments/1bpgrdf/wanted_... I've got about a dozen people signed up. Just working through some hardware issues right now (see above about high rate of failures), and hope to have this resolved next week, so that I can get people onto them and doing their testing.

Shiny! You have you guys gotten MI300X's working for non-inference use cases?

We got them, then gave them to a customer to use for a week, got them back and now we are having some hardware issues that we are in the process of sorting out. I literally haven't had more than a few hours time on them yet.

They _should_ work fine for both training and inference, but since nobody has done much in the way of public in-depth benchmarks yet... I was hoping to get people to do it for us in order to stay as unbiased as possible.

noticing now: strange that my previous comment was downvoted. Would be nice to understand what someone didn't like about what I said!

Re: Groq CEO: 'We No Longer Sell Hardware'

#64

Post by LoganDark »

wmf wrote: post #46
LoganDark wrote: post #24

That sucks. I wanted to save up for a couple years and get some hardware for home, but I guess the "AI" space moves so fast you barely get a couple months

Save up for Tenstorrent instead.

I'll look into it, though seeing "contact us" always makes me think they're not going to sell a single unit to a home user. (With that said, Groq probably wouldn't either. You can technically buy LPUs for 20k each, without an expectation of support, but it takes tens of them to run Mixtral.)

Tenstorrent also looks incredibly Python-specific (as in, everything including their SMI seems mostly Python-based) which doesn't seem promising?

Re: Groq CEO: 'We No Longer Sell Hardware'

#65

Post by ilaksh »

I'm not able to get consistent replies from the API. It's lightening fast for like ten minutes and then starts freezing up for several seconds.

I want to use it, but it's been very unreliable. I have been using Claude 3 and thinking about together.ai with Mixtral.

Re: Groq CEO: 'We No Longer Sell Hardware'

#66

Post by latchkey »

ethbr1 wrote: post #57
aleph_minus_one wrote: post #27

Earlier quoted context omitted.

> If you have to sell the hardware and support people buying it, that is a world of trouble. What is the difference between this and having to sell the cloud access and supporting the people who buy a subscription?

> What is the difference between this and having to sell the cloud access and supporting the people who buy a subscription Knowledge/training. If you're shipping a brand new hardware arch, exposed as raw hardware, then you're on the hook for training everyone in the world and fixing all their weird edge case uses. I.e. are you willing to invest in Intel/AMD/Nvidia-scale QA and support? If you're exposing a PaaS (or e…

> I.e. are you willing to invest in Intel/AMD/Nvidia-scale QA and support?

At least in our experience, the first line of support is from the chassis vendors. You don't go to a store and buy MI300x. You buy them from someone like SMCI/Dell, who provides the support. Of course, behind the scenes, they might be talking to AMD. Even those chassis companies often have other providers of their gear (like Exxact) as another line of defense as well.

In the case of Groq, it would have been death by 1000 cuts to have to support end users directly, especially if they are selling small quantities. It is much easier to just build data centers full of gear, maintain it yourself and then just rent the time on the hardware.

Re: Groq CEO: 'We No Longer Sell Hardware'

#67

Post by sipjca »

LoganDark wrote: post #64
wmf wrote: post #46

Earlier quoted context omitted.

Save up for Tenstorrent instead.

I'll look into it, though seeing "contact us" always makes me think they're not going to sell a single unit to a home user. (With that said, Groq probably wouldn't either. You can technically buy LPUs for 20k each, without an expectation of support, but it takes tens of them to run Mixtral.) Tenstorrent also looks incredibly Python-specific (as in, everything including their SMI seems mostly Python-based) which doesn…

fwiw as a consumer I have a tenstorrent card in my machine

Re: Groq CEO: 'We No Longer Sell Hardware'

#68

Post by ukd1 »

latchkey wrote: post #14

Interesting, I guess that is why I never got a response back from them about buying their stuff. My guess is that they realized that just selling hardware is a lot harder than running it themselves. Deploying this level of compute is non-trivial, with very high rates of failure, as well as huge supply chain issues. If you have to sell the hardware and support people buying it, that is a world of trouble. > no-one wan…

How do y'all compare to https://tensorwave.com?

Re: Groq CEO: 'We No Longer Sell Hardware'

#69

Post by latchkey »

ukd1 wrote: post #68
latchkey wrote: post #14

Interesting, I guess that is why I never got a response back from them about buying their stuff. My guess is that they realized that just selling hardware is a lot harder than running it themselves. Deploying this level of compute is non-trivial, with very high rates of failure, as well as huge supply chain issues. If you have to sell the hardware and support people buying it, that is a world of trouble. > no-one wan…

How do y'all compare to https://tensorwave.com ?

To be totally honest, I have no idea. When I first learned about them, I reached out to the CEO privately on LinkedIn, he asked what I was up to, I told him probably more than I should have (I come from an open source and transparent background), then he stopped talking to me entirely.

Since then, one of the co-founders blocked me on Twitter for pointing out that despite their claims, they were not the first MI300x to production. Neither were we, ElioVP gets that trophy, then Lamini, then GigaIO. Making us 4th and them 5th. I could go on and on with weird stuff I've seen them do, but it just isn't productive here.

Anyway, I think we have some overlap since we both are one of the few startups on the planet that actually has MI300x. But beyond that, I believe strongly that this space is large enough for multiple players and I don't see a need to be weird with each other. Apparently, I'm not on the same page though.

¯\_(ツ)_/¯

Re: Groq CEO: 'We No Longer Sell Hardware'

#70

Post by QuadrupleA »

ilaksh wrote: post #65

I'm not able to get consistent replies from the API. It's lightening fast for like ten minutes and then starts freezing up for several seconds. I want to use it, but it's been very unreliable. I have been using Claude 3 and thinking about together.ai with Mixtral.

Same, it's great when it's quick / available, but they seem underprovisioned for busy times and I often get long 10-30 second stalls.
Post reply on HN