81–90 of 131 posts
I can't wait until LLMs are fast enough that a single response can actually be a whole tree of thought/review process before giving you an answer, yet is still fast enough to not even notice
Earlier quoted context omitted.
They addressed it on their blog https://groq.com/hey-elon-its-time-to-cease-de-grok/
can you trademark a verb ?
Lots of comments talking about the model itself. This is Llama 2 70B, a model that has been around for a while now, so we're not seeing anything in terms of model quality (or model flaws) we haven't seen before. What's interesting about this demo is the speed at which it is running, which demonstrates the "Groq LPU™ Inference Engine". That's explained here: https://groq.com/lpu-inference-engine/ > This is the world’s…
this is running on custom hardware, if you’re curious about the underlying architecture check the publication below. https://groq.com/wp-content/uploads/2023/05/GroqISCAPaper202... EDIT: i work at Groq, but i’m commenting in a personal capacity. happy to answer clarifying questions or forward them along to folks who can :)
In case, it's not blinding obvious to people. Groq are a hardware company that have built chips that are designed around the training and serving of machine models particularly targeted at LLMs. So the quality of the response isn't really what we're looking for here. We're looking for speed i.e. tokens per second. I actually have a final round interview with a subsidiary of Groq coming up and I'm very undecided as to…
But if it was generating high-quality responses, would that not make it go slower?
Earlier quoted context omitted.
fast but wrong/gibberish
Its using vanilla llama-2 from Meta with no fine tuning. The point here is the speed and responsiveness of the underlying HW and SW.
Yeah, it’s fast but almost always wrong. I asked it a few things (recipes, trivia etc…) and it completely made up the answers. These things don’t really know how to say “I don’t know” and pretend to know everything.
Can you provide specifics for what you asked what it answered? It seems to answer my questions, including recipes correctly.
What are some relevant speed metrics? Output tokens per second? How about number of input tokens -- does that matter/how does that factor in.
Another censored and boring Google reader. It lied to me twice in 4 prompts and was forced to apologise when called out. Am I wrong in thinking that the first company to develop an unfiltered and genuine intelligence is going to win this AI game?
In case, it's not blinding obvious to people. Groq are a hardware company that have built chips that are designed around the training and serving of machine models particularly targeted at LLMs. So the quality of the response isn't really what we're looking for here. We're looking for speed i.e. tokens per second. I actually have a final round interview with a subsidiary of Groq coming up and I'm very undecided as to…