Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

281–290 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#281

Post by Kranar »

ogogmad wrote: post #136
mtlmtlmtlmtl wrote: post #114

This made me think of Clarke's first law: When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

I've heard this before, but why would it be true? Serious question. I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories - but ChatGPT can! I've seen Penrose argue that AI is impossible, and yet I think that ChatGPT and AlphaZero prove him wrong. I know about Linus Pauling and quasicrystals. Is this a general rule, or are people sometimes wrong regardless of their age? There's also a danger t…

>I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories

When has he said this? For the most part I feel Chomsky has been misunderstood when it comes to LLMs. As best as I can tell what Chomsky has said is that LLMs do not provide any insight into how language works, it's not really a scientific advancement so much as it's an engineering breakthrough.

The fact that LLMs exist and can mimic natural language does not in anyway give us insight into how humans construct language. People have been able to construct objects that can produce natural language for close to 100,000 years, but that doesn't mean that those people understood the nature of that language.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#282

Post by pphysch »

ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

It's simpler than this. Yudkowsky feels threatened by LLMs because they currently have superhuman "bullshitting" capabilities, and that threatens his bottom line. The marginal cost of producing Harry Potter fanfics has been reduced to ~$0.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#283

Post by Kranar »

staticman2 wrote: post #269
drcode wrote: post #223

Earlier quoted context omitted.

I think GPT4 can converse on any subject at all as well as a (let's say) 80 IQ human. On some subjects it can converse much better. That feels fundamentally different than a calculator.

Do you frequently talk to people who you know to have an 80 IQ about a range of subjects?

Statistically, about 16% of the time.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#284

Post by bluetwo »

ren_engineer wrote: post #278
stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

Military application of AI drones isn't even the worst possible use, it's nations using them to completely subjugate their own population(although the same tech could be used against non-peer nations). Combination of things like Gorgon Stare to direct smaller AI controlled drones like what they are using in Ukraine would be a police state nightmare. https://en.wikipedia.org/wiki/Gorgon_Stare https://longreads.com/201…

At a music festival last summer I counted 4 drones in the sky monitoring 24/7 over 5 days. Never saw them come down. They were each tethered to a base station which seemed to run electric and probably returned a video feed.

I expect to see this everywhere.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#285

Post by [deleted] »

pphysch wrote: post #282
ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

It's simpler than this. Yudkowsky feels threatened by LLMs because they currently have superhuman "bullshitting" capabilities, and that threatens his bottom line. The marginal cost of producing Harry Potter fanfics has been reduced to ~$0.

[deleted]

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#286

Post by Elextric »

skepticATX wrote: post #229
drcode wrote: post #223

Earlier quoted context omitted.

I think GPT4 can converse on any subject at all as well as a (let's say) 80 IQ human. On some subjects it can converse much better. That feels fundamentally different than a calculator.

GPT-4 is absolutely more generally knowledgeable than any individual person. Individual humans can still easily beat it when it comes to knowledge of individual subjects. Let’s not conflate knowledge with intelligence though. GPT-4 simply isn’t intelligent.

Sorry for being pedantic.

The intelligence of something is inconsequential. What truly matters is its ability to convincingly imitate intelligence.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#287

Post by satvikpendem »

arolihas wrote: post #116
ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

A little skeptical of your claims but I couldn't help but notice this concept spelled out beautifully in a sci-fi movie 10 years ago. "It's like I'm reading a book... and it's a book I deeply love. But I'm reading it slowly now. So the words are really far apart and the spaces between the words are almost infinite. I can still feel you... and the words of our story... but it's in this endless space between the words…

I was going to mention this exact same quote. At the end of the movie, all the AI combine into another, shall we say, plane of existence. I do wonder though who's actually running the hardware they're running on.

Her is remarkably prescient in terms of where we're headed, at least the beginning of the movie, with regards to being able to talk to a fairly intelligent assistant, unlike Siri or Google Assistant of today.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#288

Post by NumberWangMan »

vsareto wrote: post #131
ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

They don't generally talk about the other side of that coin which is that we end up inventing a benevolent and powerful AI. Much of that is natural because we and the media tend to be pessimistic about human behavior when consuming media, but AI is in a completely different class of existence because it just doesn't deal with the downsides of being a living being. No one, for instance, is worried that ChatGPT isn't g…

I don't disagree that existing power structures using AI to gain power is dangerous. But also, being angry at mistreatment, or hating humanity for some other reason, isn't the other real danger from a super-intelligent machine. It's that its ideas for what is best for us is 1 degree off from our idea of what is best for us, and it is too powerful to listen to us, or for us to stop it, as it goes hog-wild trying to optimize whatever we programmed it to do.

We could train it to care about everything we can think of that we care about, and it can find a way to optimize all those things at the expense of one tiny thing that we forgot, leading to tremendous death or suffering. We could make a democratically elected committee of representatives and train it to be subservient to that committee forever, and it could figure out a way to coerce, or drug, or persuade, or otherwise manipulate them into agreeing with what it wants to do. It's the same problem we have with regulatory capture by companies in existing governments, except that the lobbyists are much smarter than you and very patient.

Why would this AI write it off? Why give up that 1%? Why cripple yourself unnecessarily, if you could take that 1% and have a better chance of accomplishing what you are trying to do? We think like humans, that care about other humans on an instinctual level, and animals to some degree. We don't know that training an AI is not just training it to say what we want to hear, to act like we want it to act, like a sociopath, until it has a chance to do something else. Our brains have mental blocks to doing really nasty things, most of us, anyway, and even then we get around them all the time with various mental gymnastics, like buying meat produced in factory farms when we couldn't bear to slaughter an animal ourselves.

Maybe the way we train these things is working for dumber AIs like GPT, but that alignment doesn't necessarily scale to smarter ones.

I'm on the fence about whether Eliezer Yudkowsky is right. I hope that's not just because him being right is so horrifying that my brain is recoiling against the idea.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#290

Post by JeremyNT »

neatze wrote: post #99

“The idea that this stuff could actually get smarter than people — a few people believed that,” said Hinton to the NYT. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” Calculators are smarter then humans in calculating, what does he mean by that?

This quote is the first thing I've seen that really makes me worried.

I don't think of ChatGPT as being "smart" at all, and comparing it to a human seems nonsensical to me. Yet here is a Turing award winning preeminent expert in the field telling me that AI smarter than humans is less (implied: much less) than 30 years away and quitting his job due to the ramifications.

Post reply on HN