Live data from Hacker News

SambaNova chip for LLMs nearly as fast as Groq

fast.snova.ai

11–12 of 12 posts

Re: SambaNova chip for LLMs nearly as fast as Groq

#11

This offers no insight into the architecture of either system or the chips they developed. Also, why would I want to access their stack and have them use my queries as training data? I see nothing from either Groq or SambaNova that says that they will distribute dedicated inference chips in any other form than full data centers. If I can't slot it into my machine, is it real? How exactly are these companies envisioni…

Let me know how that goes.

Re: SambaNova chip for LLMs nearly as fast as Groq

#12
post #6

Earlier quoted context omitted.

Both are faster than NVIDIA! Always good to see competition!

I mean that 'almost as fast as Groq' is not a compelling strategy. If it's almost as fast, why not just go with Groq? Are they going for cheaper? Added services? I'm just wondering.

Groq quantize their models, killing accuracy. SambaNova run the models at full precision, and on a single node, whilst Groq runs a single model on hundreds of chips. SambaNova run training and inference, Groq only run inference. SambaNova run hundreds on models on a single node without any performance degradation, Groq need hundreds of chips for a single model.

Groq are trying to launch their own cloud and running it at a loss to try to acquire users are raise venture capital so they can tape out a new chip (their current chip just is not very competitive, they get speed for having a good interconnect, but it is so cost ineffective). SambaNova sell systems to big companies and service providers, and don’t have a paid cloud API, so aren’t as visible as Groq. But SambaNova make real revenue whilst Groq have literally zero revenue, which is why they struggle to raise money.

Very soon people will realize that Groq are frauds and SambaNova is the legit challenger to Nvidia.

Post reply on HN