11–19 of 19 posts
Wow.. I wonder how this compares to groq? I made a request to access their developer cloud. Anyone have any idea when they start processing those requests and how many slots they might have?
The more the merrier. If we do need power plants worth of juice then it may as well be dedicated custom hardware to optimise this to the max
Wow.. I wonder how this compares to groq? I made a request to access their developer cloud. Anyone have any idea when they start processing those requests and how many slots they might have?
There are a couple now. Groq , sambanova and tenstorrent come to mind. The more the merrier. If we do need power plants worth of juice then it may as well be dedicated custom hardware to optimise this to the max
Earlier quoted context omitted.
There are a couple now. Groq , sambanova and tenstorrent come to mind. The more the merrier. If we do need power plants worth of juice then it may as well be dedicated custom hardware to optimise this to the max
Thanks for telling me about Sambanova. Too bad they don't have an API. The speed is absolutely incredible. They actually have a demo where it just keeps re-running the chat completion as you are typing your query.
Think they do, but I couldn't sign up...the verify email just never arrived and eventually got fed up.
Earlier quoted context omitted.
Thanks for telling me about Sambanova. Too bad they don't have an API. The speed is absolutely incredible. They actually have a demo where it just keeps re-running the chat completion as you are typing your query.
>Too bad they don't have an API. Think they do, but I couldn't sign up...the verify email just never arrived and eventually got fed up.
Wow.. I wonder how this compares to groq? I made a request to access their developer cloud. Anyone have any idea when they start processing those requests and how many slots they might have?
Wow.. I wonder how this compares to groq? I made a request to access their developer cloud. Anyone have any idea when they start processing those requests and how many slots they might have?
A coworker of mine recently got access to Groq. Not sure how many slots there are but we have a pretty strict rate limit for what thats worth
> Specialized chips are inevitable I mean, are they? Seems like the industry would prefer these things to become commodities, especially if it helps with portability and reproducibility.
Of course GPUs (and CPUs, and pre-transformer/LLM type NPU/TPUs) are also going to respond to better suit LLM workloads. So we may see a convergence in capabilities.
What is for sure is that the LLM chip architecture is not yet settled. Currently available chips are mostly designed pre LLM craze, with slight adaptations. And have large areas of potential improvement, at least 10-100x (maybe 1000x) in power efficiency. And power efficiency is a key element (in addition to up from hardware investment). Which is the opportunity this startup (and others) have identified.
> Specialized chips are inevitable I mean, are they? Seems like the industry would prefer these things to become commodities, especially if it helps with portability and reproducibility.