Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Groqchat

chat.groq.com

Post reply on HN
Print view View original

71–80 of 131 posts

Re: Groqchat

#71

Post by jkachmar »

moneywoes wrote: post #59
jkachmar wrote: post #32

Earlier quoted context omitted.

this is running on custom hardware, if you’re curious about the underlying architecture check the publication below. https://groq.com/wp-content/uploads/2023/05/GroqISCAPaper202... EDIT: i work at Groq, but i’m commenting in a personal capacity. happy to answer clarifying questions or forward them along to folks who can :)

what’s the cost?

right now we’re providing this access to public, anonymous users via this demo chat interface as an alpha test.

we’ll be publishing information about API access, and pricing, shortly after the new year.

Re: Groqchat

#73

Post by agildehaus »

orenlindsey wrote: post #35

That's really fast. But it mostly seems to be because they made a custom chip. I want to see an LLM that is so highly optimized that it runs at this speed on more normal hardware.

That'd be nice, but we could also just make this hardware normal.

Re: Groqchat

#74

Post by throwaway20222 »

The censorship levels are off the charts; I am at a basketball game with my wife who is ethnically Chinese. I asked for an image of a Chinese woman dunking a basketball. I was told not only is this inappropriate, but also unrealistic and objectifying.

Re: Groqchat

#75

Post by Unbefleckt »

Another censored and boring Google reader. It lied to me twice in 4 prompts and was forced to apologise when called out. Am I wrong in thinking that the first company to develop an unfiltered and genuine intelligence is going to win this AI game?

Re: Groqchat

#76

Post by Unbefleckt »

throwaway20222 wrote: post #74

The censorship levels are off the charts; I am at a basketball game with my wife who is ethnically Chinese. I asked for an image of a Chinese woman dunking a basketball. I was told not only is this inappropriate, but also unrealistic and objectifying.

A Chinese woman dunking is unrealistic? I'm amazed that got through the filter.

Re: Groqchat

#78

Post by mlazos »

jkachmar wrote: post #32
simonw wrote: post #26

Lots of comments talking about the model itself. This is Llama 2 70B, a model that has been around for a while now, so we're not seeing anything in terms of model quality (or model flaws) we haven't seen before. What's interesting about this demo is the speed at which it is running, which demonstrates the "Groq LPU™ Inference Engine". That's explained here: https://groq.com/lpu-inference-engine/ > This is the world’s…

this is running on custom hardware, if you’re curious about the underlying architecture check the publication below. https://groq.com/wp-content/uploads/2023/05/GroqISCAPaper202... EDIT: i work at Groq, but i’m commenting in a personal capacity. happy to answer clarifying questions or forward them along to folks who can :)

How many chips are used for this demo? Do they have dram? I remember the earlier versions did not have dram.

Are they also used for training or just inference?

Re: Groqchat

#79

Post by m1sta_ »

jkachmar wrote: post #32
simonw wrote: post #26

Lots of comments talking about the model itself. This is Llama 2 70B, a model that has been around for a while now, so we're not seeing anything in terms of model quality (or model flaws) we haven't seen before. What's interesting about this demo is the speed at which it is running, which demonstrates the "Groq LPU™ Inference Engine". That's explained here: https://groq.com/lpu-inference-engine/ > This is the world’s…

this is running on custom hardware, if you’re curious about the underlying architecture check the publication below. https://groq.com/wp-content/uploads/2023/05/GroqISCAPaper202... EDIT: i work at Groq, but i’m commenting in a personal capacity. happy to answer clarifying questions or forward them along to folks who can :)

How easy is it for companies to setup private local servers using Grow hardware (cost and complexity). I've got money. I want throughout.
Post reply on HN