Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Groqchat

chat.groq.com

Post reply on HN
Print view View original

1–10 of 131 posts

Re: Groqchat

#3

Post by badFEengineer »

This was surprisingly fast, 276.27 T/s (although Llama 2 70B is noticeably worse than GPT-4 turbo). I'm actually curious if there's good benchmarks for inference tokens per second- I imagine it's a bit different for throughput vs. single inference optimization, but curious if there's an analysis somewhere on this

edit: I re-ran the same prompt on perplexity llama-2-70b and getting 59 tokens per sec there

Re: Groqchat

#4

Post by andreagrandi »

Yeah, it’s fast but almost always wrong. I asked it a few things (recipes, trivia etc…) and it completely made up the answers. These things don’t really know how to say “I don’t know” and pretend to know everything.

Re: Groqchat

#5

Post by Mockapapella »

I can't wait until LLMs are fast enough that a single response can actually be a whole tree of thought/review process before giving you an answer, yet is still fast enough to not even notice

Re: Groqchat

#6

Post by MH15 »

Incredibly fast. I wonder if they've released verification that it matches llama 70b on regular hardware?

Re: Groqchat

#7

Post by axus »

I asked "How up to date is your information about the world?"

It said December 2022, but the answers to another question was not correct for that time or now. It also went into some kind of repeating loop to its maximum response length.

Still pretty cool that our standards for chat programs have risen.

Re: Groqchat

#8

Post by phildenhoff »

It’s very fast at telling me it can’t tell me things!

I asked about creating illicit substances — an obvious (and reasonable) target for censorship. And, admirably, it suggested getting help instead. That’s fine.

But I asked for a poem about pumping gas in the style of Charles Bukowski, and it moaned that I shouldn’t ask for such mean-spirited, rude things. It wouldn’t dare create such a travesty.

Re: Groqchat

#9

Post by givemeethekeys »

andreagrandi wrote: post #4

Yeah, it’s fast but almost always wrong. I asked it a few things (recipes, trivia etc…) and it completely made up the answers. These things don’t really know how to say “I don’t know” and pretend to know everything.

Can you provide specifics for what you asked what it answered? It seems to answer my questions, including recipes correctly.
Post reply on HN