Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

361–370 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#361

Post by scarmig »

moonchrome wrote: post #343
belter wrote: post #297

The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe -- ninety-six billion planets -- into the supercircuit that would connect them all into one supercalculator, one cybernetics machine that would combine all the knowledge of all the galaxies. Dwar Reyn spoke briefly to the watching and listening trillions. Then after a moment's silence he said…

I wonder at what point does alingment become an issue for AI systems ? Given sufficiently large distances, assuming no FTL communication, if you're spawning copies with the same goals you're risking misalignment and creating equally powerful adversaries outside of your light cone.

Assuming no FTL, powerful adversaries outside your light cone are unable to do anything to you.

Though it remains possible that latency between components in an AI system could become so large that it couldn't enforce consistency between.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#362

Post by [deleted] »

tivert wrote: post #167
dlkf wrote: post #6

Earlier quoted context omitted.

Cade Metz is the same hack who tried to smear Scott Alexander. This guy is the personification of journalistic malpractice.

> Cade Metz is the same hack who tried to smear Scott Alexander. This guy is the personification of journalistic malpractice. He didn't "smear" Scott Alexander. That's just the hit-job framing pushed by Alexander's fans, who were mad he didn't write a puff piece and they couldn't just make up rules on about stuff on their websites (e.g. about using people's self-disclosed real names) and have the rest of the world be…

[deleted]

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#363

Post by ska »

dist-epoch wrote: post #193
ogogmad wrote: post #136

Earlier quoted context omitted.

I've heard this before, but why would it be true? Serious question. I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories - but ChatGPT can! I've seen Penrose argue that AI is impossible, and yet I think that ChatGPT and AlphaZero prove him wrong. I know about Linus Pauling and quasicrystals. Is this a general rule, or are people sometimes wrong regardless of their age? There's also a danger t…

The usual explanation is that they will call impossible something which goes against their life's work because in their mind it nullifies it, while a youngster has less or zero "sunken cost". A related saying: "science advances a funeral at a time", meaning the old-guard blocks new theories for the same reason, they go against their life's work.

This is true, but misses the important part that they (the older set) are often correct. For every new idea that really changes everything there are a huge number that die on the vine or just become a ho-hum tool in a big toolbox.

Most new ideas are less interesting and impactful than they seem when you are in the middle of their creation. You never really get to see what's happening until much much later.

A variant of all this is that you should trust the old guard when they tell you something can be done, but not when they tell you it can't. There is a good quote about that I've forgotten.

The corollary is that you shouldn't really trust the young turks on anything, but you should support their efforts and test the results.

It's very human to see yourself as Planck in the early 1900s not Wolfram in the early 2000s.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#364

Post by moonchrome »

skulk wrote: post #358
moonchrome wrote: post #343

Earlier quoted context omitted.

I wonder at what point does alingment become an issue for AI systems ? Given sufficiently large distances, assuming no FTL communication, if you're spawning copies with the same goals you're risking misalignment and creating equally powerful adversaries outside of your light cone.

> creating equally powerful adversaries outside of your light cone. If we're still bound by special relativity (since you said no FTL), there is nothing outside of your light cone.

I meant in the time it would take to synchronize/align

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#365

Post by timschmidt »

drawnwren wrote: post #322
lumost wrote: post #158

Earlier quoted context omitted.

There is such a blurry line for autonomous munitions. militaries used dumb imprecise munitions for decades - then precision weapons. A2A missiles used to lock on radar signature leading to huge risks related to accidentally shooting airliners/friendly craft. Now antiship missiles dynamically select their target over 300km away to maximize the chance of hitting a big ship. During the war on terror, ML models would dec…

This comment appears to be getting upvotes, so I am going to jump in and say that it is very clear to anyone who knows that this commenter has no idea how AI is being used in the military or how military targeting works in the West. These things are generally classified, but algorithms are nowhere close to being considered decision-capable and human-in-the-loop targeting will be the norm for quite some time. Even nex…

"SKYNET is a program by the U.S. National Security Agency that performs machine learning analysis on communications data to extract information about possible terror suspects. The tool is used to identify targets, such as al-Qaeda couriers, who move between GSM cellular networks. Specifically, mobile usage patterns such as swapping SIM cards within phones that have the same ESN, MEID or IMEI number are deemed indicative of covert activities.[1][2] Like many other security programs, the SKYNET program uses graphs that consist of a set of nodes and edges to visually represent social networks.[3] The tool also uses classification techniques like random forest analysis. Because the data set includes a very large proportion of true negatives and a small training set, there is a risk of overfitting.[1] Bruce Schneier argues that a false positive rate of 0.008% would be low for commercial applications where "if Google makes a mistake, people see an ad for a car they don't want to buy" but "if the government makes a mistake, they kill innocents."[1]

The SKYNET project was linked with drone systems, thus creating the potential for false-positives to lead to deaths.[1][5]"

https://en.wikipedia.org/wiki/SKYNET_(surveillance_program)

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#366

Post by moonchrome »

scarmig wrote: post #361
moonchrome wrote: post #343

Earlier quoted context omitted.

I wonder at what point does alingment become an issue for AI systems ? Given sufficiently large distances, assuming no FTL communication, if you're spawning copies with the same goals you're risking misalignment and creating equally powerful adversaries outside of your light cone.

Assuming no FTL, powerful adversaries outside your light cone are unable to do anything to you. Though it remains possible that latency between components in an AI system could become so large that it couldn't enforce consistency between.

>Though it remains possible that latency between components in an AI system could become so large that it couldn't enforce consistency between.

Yeah that's what I was trying to say - if they are far enough to synchronize/enforce consensus you basically have to assume they could be hostile in every future interaction.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#367

Post by janalsncm »

chrsjxn wrote: post #315
neatze wrote: post #99

“The idea that this stuff could actually get smarter than people — a few people believed that,” said Hinton to the NYT. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” Calculators are smarter then humans in calculating, what does he mean by that?

That statement seems like such science fiction that it's kind of baffling an AI expert said it. What does it even mean for the AI to be smarter than people? I certainly can't see a way for LLMs to generate "smarter" text than what's in their training data. And even the best case interactions I've seen online still rely on human intelligence to guide the AI to good outcomes instead of bad ones. Writing is a harder tas…

Totally agreed that words like “smart” and “intelligent” are loaded and poorly defined. Competence is a better term since it implies some sort of metric has been used to compare to humans.

However, even at human levels of competence a tool can be superior by being faster or more scalable than humans.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#368

Post by fatherzine »

intalentive wrote: post #337

We still don’t have Rosie the Robot. When it comes to learning and adapting to new environments, we don’t even have AI as smart as a mouse. LeCun is right, there is still a long way to go.

"When it comes to learning and adapting to new environments, while we are lucky AI's aren't yet as smart as a mouse, they are uncomfortably close, and the pace of progress is unnerving. Hinton is right, we've got too far and we should grind all AI research to a halt via heavy global regulation."

What is the goal here? Creation of an all powerful God? Self-destruction as a species? I'm not up-to-date with the exact state of the AI research, or with various AI luminaries position nuances, but I can read a first-principles back-of-the-envelope chart. It doesn't look good, especially for a committed speciist like myself.

Edit. The signs are of a very serious situation. Experts are ringing the alarm of massive scale societal disruption and possibly destruction left and right. While we may not be able to do anything about it, perhaps we could act a little less callous about it.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#369

Post by vintermann »

drawnwren wrote: post #322
lumost wrote: post #158

Earlier quoted context omitted.

There is such a blurry line for autonomous munitions. militaries used dumb imprecise munitions for decades - then precision weapons. A2A missiles used to lock on radar signature leading to huge risks related to accidentally shooting airliners/friendly craft. Now antiship missiles dynamically select their target over 300km away to maximize the chance of hitting a big ship. During the war on terror, ML models would dec…

This comment appears to be getting upvotes, so I am going to jump in and say that it is very clear to anyone who knows that this commenter has no idea how AI is being used in the military or how military targeting works in the West. These things are generally classified, but algorithms are nowhere close to being considered decision-capable and human-in-the-loop targeting will be the norm for quite some time. Even nex…

Well, then I am going to jump in to say that insiders who jump in to assure us that due to their classified insider knowledge they can reassure us it's not like that at all... you realize this doesn't work, right? Not on anyone who doesn't already believe it. Even if you were right, you're simply not trustworthy.

There are two things I wish I could make you and your friends understand: one is how deep you've dug that credibility hole. There was a price to pay for all that secrecy.

The other is, when you're out here trying to manage the non-classified public's perceptions, why on earth would you trust that your bosses aren't managing yours? Why would you think you get to know what the long term plans are?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#370

Post by lxe »

Disappointing. Yet another industry leader sewing public FUD for some reason. Why not bring rational discourse into the conversation around software safety and ethics.

Automation has been the driving force of industry since the industrial revolution itself. We're not new to automation, and we are certainly not new to safety of autonomous systems. AI is no different.

Post reply on HN