Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

221–230 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#221

Post by flangola7 »

yowzadave wrote: post #88
politician wrote: post #81

After reading the NYT interview, I don’t understand why he still chose to invent, in his words, a dangerous technology and publish the results openly. Not a criticism of the man, but of the article.

I have this same question about the (apparently many) AI researchers who believe it poses significant risks to humanity, yet still push forward developing it as fast as they can.

I recently listened to a journalist who spoke to many AI workers in SV. There is an alarmingly pervasive pseudo-religious attitude that they are ushering in a new form of life, and that it is their destiny to be the people who make it happen. Most of them will candidly tell you humanity may go extinct as a result (with at least a 1 in 10 chance), but they choose to plow ahead regardless.

Others appear to be in common modes of willful denial: hubris or salary-depends-on-not-knowing syndrome.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#222

Post by neatze »

lumost wrote: post #158
stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

There is such a blurry line for autonomous munitions. militaries used dumb imprecise munitions for decades - then precision weapons. A2A missiles used to lock on radar signature leading to huge risks related to accidentally shooting airliners/friendly craft. Now antiship missiles dynamically select their target over 300km away to maximize the chance of hitting a big ship. During the war on terror, ML models would dec…

The line already drawn, no indiscriminate killings of civilians, all this technology only minimizes civilian suffering, furthermore it is more humane to use terminators instead of 20 years olds clearing buildings and autonomous tanks/tracks driving though IEDs filled streets.

War is deeply in human nature, it is not going away, question is; how to make it more humane and minimize indiscriminate killings and sufferings in most cases poorest populations on earth.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#223

Post by drcode »

neatze wrote: post #99

“The idea that this stuff could actually get smarter than people — a few people believed that,” said Hinton to the NYT. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” Calculators are smarter then humans in calculating, what does he mean by that?

I think GPT4 can converse on any subject at all as well as a (let's say) 80 IQ human. On some subjects it can converse much better.

That feels fundamentally different than a calculator.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#224

Post by pmoriarty »

lucumo wrote: post #189
sudhirj wrote: post #139

Earlier quoted context omitted.

Yeah, this seems like more of a problem than vague statements about AGI. We're still in the scope of ML - ChatGPT can't play chess, for example, and a self driving model can't write Haiku. An AGI would be able to do all of them. It seems much more likely that a fleet of autonomous (in the name of cutting costs) war machines will be created with relatively simple ML models that work in intended (or otherwise) ways to…

> ChatGPT can't play chess, for example Out of pure curiosity I tried this a while back. At some point it tried to move a knight from d4 to f6, and it was the most stubborn and obstinate child when I tried to correct it. It was the best fun I had playing chess in a few decades. Excerpt: _Me:_ Can a knight move from f6 to d4? _ChatGPT:_ Yes, a knight can move from f6 to d4, as long as there are no pieces blocking its…

I tried this question on various LLMs on poe.com and Claude, Sage, and Claude+ correctly answered that a knight can't move from f6 to d4.

Dragonfly failed the test, and though Claude did answer the question correctly, it gave some other examples of invalid moves for a knight on f6. All of the moves that Claude+ gave were valid, but it didn't list all possible moves for a knight on f6.

I didn't try GPT4.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#225

Post by drcode »

ttul wrote: post #164
0zemp2c wrote: post #122

countdown until he starts his own AI company and gets hundreds of millions in seed investment... can't say I blame him, everyone in AI who can make a cash grab should do so

Nah, Hinton is already incredibly rich. His first startup was bought by Google for $44M. And Google paid him millions more for a decade. Dr. Hinton is in a rare position of having no reason to work for anyone, not even venture capitalists.

Sam Altman was rich and stated concern for AI x-risk when he started OpenAI

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#226

Post by pmoriarty »

jstanley wrote: post #188
sudhirj wrote: post #139

Earlier quoted context omitted.

Yeah, this seems like more of a problem than vague statements about AGI. We're still in the scope of ML - ChatGPT can't play chess, for example, and a self driving model can't write Haiku. An AGI would be able to do all of them. It seems much more likely that a fleet of autonomous (in the name of cutting costs) war machines will be created with relatively simple ML models that work in intended (or otherwise) ways to…

I played chess against ChatGPT just yesterday, and it got into a winning position against me. After 24 moves it tried to play an illegal move and then when I told it that's illegal it played a bad move, and after that it didn't manage to find any more legal moves (I gave up after asking it to try again about 10 times). But it's very close to being able to play chess. My prompt was: > we're playing chess. only send ch…

Did you repeat the board position back to it after each move? LLMs have a limited context, so they might forget the board position after a while, unless they're reminded.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#227

Post by neatze »

renewiltord wrote: post #216
neatze wrote: post #99

“The idea that this stuff could actually get smarter than people — a few people believed that,” said Hinton to the NYT. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” Calculators are smarter then humans in calculating, what does he mean by that?

Sibling comment is correct to prompt you to at least try an LLM first. It's unfortunately the equivalent of lmgtfy.com but it's true.

What makes you think I did not try, simply fail to see why/how natural language inconstant comprehension in any way equates to human or any other animal behavior, I simply don't believe/see (subjectively) that any potential of prompt hacking with massive datasets will build consistent anticipatory system (planning and some aspect of learning).

As analogy, the more I look at it, the more it looks like an geocentric model of solar system.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#228

Post by [deleted] »

lumost wrote: post #158
stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

There is such a blurry line for autonomous munitions. militaries used dumb imprecise munitions for decades - then precision weapons. A2A missiles used to lock on radar signature leading to huge risks related to accidentally shooting airliners/friendly craft. Now antiship missiles dynamically select their target over 300km away to maximize the chance of hitting a big ship. During the war on terror, ML models would dec…

[deleted]

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#229

Post by skepticATX »

drcode wrote: post #223
neatze wrote: post #99

“The idea that this stuff could actually get smarter than people — a few people believed that,” said Hinton to the NYT. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” Calculators are smarter then humans in calculating, what does he mean by that?

I think GPT4 can converse on any subject at all as well as a (let's say) 80 IQ human. On some subjects it can converse much better. That feels fundamentally different than a calculator.

GPT-4 is absolutely more generally knowledgeable than any individual person. Individual humans can still easily beat it when it comes to knowledge of individual subjects.

Let’s not conflate knowledge with intelligence though. GPT-4 simply isn’t intelligent.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#230

Post by meh8881 »

neilk wrote: post #170
meh8881 wrote: post #128

Earlier quoted context omitted.

Regardless of incentives, I don’t see any particular reason to think he has a more informed view than other experts on the trajectory of AI. He’s made several incorrect bets (capsule networks). I’m sure he’s smart and all. His contributions were valuable. But he’s not special in this particular moment.

Your viewpoint is fascinating. So the inventor of backpropagation, Turing award winner, Google researcher, mentor to the CTO of OpenAI, doesn’t have any special insights about AI and the tech industry that’s forming around it? He might as well be some guy off the street? Who, in your opinion, _does_ have enough context to be worth our attention? Because if you’re waiting for Sam Altman or the entire OpenAI team to sa…

Someone who is actually doing it would be a lot more authoritative in my opinion. Hinton has been wrong on most of his big ideas in the past decade. He hasn’t actually been involved in the important advances of anything recent. Inventing backprop is great. No discredit to him there. But that’s not a free pass to be seen as someone who is on the cutting edge.

But beyond all of that, what are we really asking? Are we asking about social ramifications? Because I don’t think the OpenAI devs are particularly noteworthy in their ability to divine those either. It’s more of a business question if anything. Are we talking about where the tech goes next? Because then it’s probably the devs or at least indie folks playing with the models themselves.

None of that means Hinton’s opinions are wrong. Form your own opinions. Don’t delegate your thinking.

Post reply on HN