Groq CEO: 'We No Longer Sell Hardware'

Post by latchkey » Mon, Apr 08, 2024, 1:00 AM UTC

alephnerd wrote: post #38

latchkey wrote: post #37

Earlier quoted context omitted.

The pricing isn't necessarily much higher. The pricing is set up to amortize the cost of running things over a period of time. If you're only going to use it for a short time, you pay more than someone who commits to a 3 year contract. It also isn't just the hardware capex, it is everything involved under the covers. Market pricing also factors into that as well. This is something I've struggled with myself quite a b…

> it is everything involved under the covers Yep! You explained it better than me! > Groq probably realized that people buying 1-2 cards at a time, wasn't going to be profitable Yep! And if would have bogged them down by slowing down R&D cycles and even fulfilling orders as they obviously are not placing orders the same size as Nvidia or AMD. I wonder what this portends for SambaNova and other similar vendors as well…

Post by thawab » Mon, Apr 08, 2024, 1:07 AM UTC

creato wrote: post #22

theturtletalks wrote: post #17

Earlier quoted context omitted.

If you read on, Groq said they would only sell hardware to US companies and outside companies would get cloud services, not the LPUs. I think the US government told them to keep the LPUs in-house since they could be the secret sauce for scale.

I'm not questioning the deployment strategy, I'm wondering why Saudi Aramco wants to access so much compute power that is highly specialized(?) for generative AI workloads. Or is it more general than that?

Post by wmf » Mon, Apr 08, 2024, 1:19 AM UTC

zetazzed wrote: post #13

Man, I want to appreciate a nice new hardware approach, but they say such BS that it is hard to read about them: > “There might need to be a new term, because by the end of next year we’re going to deploy enough LPUs that compute-wise, it’s going to be the equivalent of all the hyperscalers combined,” he said. “We already have a non-trivial portion of that.” Really? Does anyone seriously believe they are going to be…

Post by shrubble » Mon, Apr 08, 2024, 1:22 AM UTC

alted wrote: post #35

Custom state-of-the-art silicon is ridiculously expensive. For a minimum 100 wafers = 10k chips, Groq may have paid $100M = $10k/chip purely in amortizing design costs. Chip design (software + engineer time) and fabrication setup (lithography masks) grow exponentially [1][2] with smaller nodes, e.g., maybe $100M for Groq's current 14nm chips to ~$500M for their planned 4nm tapeout. Once you reach mass production (>>1…

Post by wmf » Mon, Apr 08, 2024, 1:23 AM UTC

rattray wrote: post #39

ipsum2 wrote: post #19

Earlier quoted context omitted.

It's not really a lie though. They require 20x more chips (storing all the weights in sram instead of hbm is expensive!) than Nvidia GPUs for a ~2x speed increase. Overall the power cost is more expensive for groq than GPUs.

Source? Or how do you know that?