Live data from Hacker News

Gopher – A 280B parameter language model

deepmind.com

31–40 of 127 posts

Re: Gopher – A 280B parameter language model

#31
post #15

The closer we get to artificial intelligence, the more we raise the bar for what qualifies as AI (as we should). Gopher/GPT-3 are already much more accurate than the average human at technical information retrieval (trivial to see from the dialogue transcripts: how many Americans know what a Schwarzschild metric is?). The focus on ethics and equity for these algorithms is interesting too, as the average human holds m…

When those language models are wrong or biased, the user will have a worse experience in all three of those scenarios. At least when we look at search results now, we can prune for the facts. Those language models are ingesting that same data to give a monolithic answer your a query. Less transparent, less safe.

"We can prune for the facts"

I wouldn't be too sure of that. People have shown time and time again that they are pretty bad at determining what a 'fact' is.

It gets far worse when they're shown more data (the issue we are dealing with right now), and these models see far more data than any one of us will see in our lifetimes.

Much of our ability to determine what a 'fact' is is either something like: (1) (if political) 'does it fit with my bias?' or (2) (if physics) 'is it physically possible?'. Since (1) is mostly dependent on what information you show yourself, and (2) depends on comparing to interaction with physical reality, the system has too much information for (1) and no physical space to run experiments in for (2). Much of our ability to determine what a 'fact' is is either something like: (1) (if political) 'does it fit with my bias?' or (2) (if physics) 'is it physically possible?'. Since (1) is mostly dependent on what information you show yourself, and (2) depends on comparing to interaction with physical reality, the system has too much information for (1) and no physical space to run experiments in for (2).

Re: Gopher – A 280B parameter language model

#33
post #15

The closer we get to artificial intelligence, the more we raise the bar for what qualifies as AI (as we should). Gopher/GPT-3 are already much more accurate than the average human at technical information retrieval (trivial to see from the dialogue transcripts: how many Americans know what a Schwarzschild metric is?). The focus on ethics and equity for these algorithms is interesting too, as the average human holds m…

> Gopher/GPT-3 are already much more accurate than the average human at technical information retrieval (trivial to see from the dialogue transcripts: how many Americans know what a Schwarzschild metric is?).

That's not a very interesting metric though. GPT-3 has access to all of wikipedia and has enough memory to store it all.

It's like saying a calculator is better at maths than a professional mathematician because it can multiply longer numbers.

Re: Gopher – A 280B parameter language model

#35

Can we please stop calling new tech "Gopher"? It's a name that belongs to a network protocol, not to a programming language or an AI model.

No. The network protocol lost cultural rights to the name by its lack of success.

The Gopher protocol was introduced in 1991, and is still in use. Are any of your projects still in use after thirty years?

Re: Gopher – A 280B parameter language model

#37

Next to "Human Expert", I'd like to see it compared to "Average American" or "Average College Grad". That might be more of a realistic notion of how close this model is to everyday US citizenry rather than experts. Sure I'd love to see a radiology assistant, too.

It might be fun for a laugh. What actual value would an AI that produces answers similar to the average person have, though? Non-expert answers for interesting questions are pretty much meaningless -- the whole point of an advanced society is that we can avoid knowing anything about most things and focus on narrow expertise.

Probably no value. I was interested in a comparison point, that is all. You can't understand how far away you are from something unless you measure it. In other words, if I asked you: how does this compare to the average person, you cannot answer because this table didn't measure it.

Re: Gopher – A 280B parameter language model

#39

Earlier quoted context omitted.

It might be fun for a laugh. What actual value would an AI that produces answers similar to the average person have, though? Non-expert answers for interesting questions are pretty much meaningless -- the whole point of an advanced society is that we can avoid knowing anything about most things and focus on narrow expertise.

Probably no value. I was interested in a comparison point, that is all. You can't understand how far away you are from something unless you measure it. In other words, if I asked you: how does this compare to the average person, you cannot answer because this table didn't measure it.

[deleted]
Post reply on HN