Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

421–430 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#421

Post by ilaksh »

NoMoreNicksLeft wrote: post #134
ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

It is absurd to think of these systems having reproductive instincts. It is so much more absurd to think that they would have these reproductive instincts not by design, but that it's some principle of intelligence itself. Natural intelligences have reproductive instincts because any organism that didn't have them built in within the first few hundred million years have no descendants for you to gawk at as they casua…

But I did not in any way say that they have reproductive instincts. Much less by accident. I agree with you.

But developers are working hard to emulate those and other artificial life characteristics explicitly in systems based on GPT and also totally different architectures.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#422

Post by runamok »

scarmig wrote: post #318
belter wrote: post #297

The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe -- ninety-six billion planets -- into the supercircuit that would connect them all into one supercalculator, one cybernetics machine that would combine all the knowledge of all the galaxies. Dwar Reyn spoke briefly to the watching and listening trillions. Then after a moment's silence he said…

That reminds me of this, more optimistically: Matter and energy had ended and with it space and time. Even AC [Automated Computer] existed only for the sake of the one last question that it had never answered from the time a half-drunken computer technician ten trillion years before had asked the question of a computer that was to AC far less than was a man to Man. All other questions had been answered, and until thi…

This is my favorite short story of all time. Thanks for including it.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#423

Post by belter »

teraflop wrote: post #407
belter wrote: post #292

My memory fails me as I read the story many years ago, and sorry already for the spoilers, but I think it's from a Philip K. Dick book. Maybe somebody here will recognize the plot and know which one it his. A Computer Science Researcher discovers AGI by accident and builds a brain that almost kills him. Spends the rest of his sad days, researching scientific articles and journal publications, that hint they are follo…

Your description doesn't match what actually happens in "Dial F for Frankenstein". The protagonists are not directly involved in creating the global network, they're just passively observing its effects, talking about it, and gradually realizing what has happened. And they don't manage to shut it down -- the story ends with them hearing news reports that militaries have lost control of their missile stockpiles, and r…

I might be misremembering it. It was more than 25 years ago. But the plot stayed imprinted to this day. I will keep looking. Only thing I am sure is that is either Philip K. Dick or Arthur C. Clarke.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#424

Post by whimsicalism »

dlkf wrote: post #6
nmstoker wrote: post #3

Hinton responded on Twitter: https://twitter.com/geoffreyhinton/status/165299357072121037...

Cade Metz is the same hack who tried to smear Scott Alexander. This guy is the personification of journalistic malpractice.

Scott Alexander needs no help in digging his own holes.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#425

Post by whimsicalism »

janalsncm wrote: post #403
EVa5I7bHFq9mnYK wrote: post #381

Earlier quoted context omitted.

It is different. The most powerful of today's machines has a red stop button. But if a machine becomes smarter than us, it could create a copy of itself without such button, so we lose control and will be quickly overpowered.

There’s an argument that we’ve gone past that point already. Yes, Microsoft can theoretically take their Bing GPT-4 program offline and turn it off, but they just invested $10B in it and they don’t want to . In fact a corporation can be thought of as an AGI itself, just made up of humans. Again, we can take Microsoft offline but we don’t want to . I guess my point is that the most likely scenario for AGI that looks m…

I see lots of people pointing to what is "more likely" or "more realistic."

I'm not sure where everyone got these strong priors on the consequences of 'intelligent' machines from, did I miss some memo?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#426

Post by thinkingemote »

nologic01 wrote: post #4

> the average person will not be able to know what is true anymore We barely held things together as society without AI unleashing cognitive noise at industrial scale. Somehow we must find ways to re-channel the potential of digital technology for the betterment of society, not its annihilation.

There's an argument that people generally do not want the truth and that AI will never be allowed to tell it. An optimist could view this as ensuring AI will be safe forever or pessimistically they might see it as AI never being authoritative ever.

One example of truth would be the topic of biological sex another about politics or economics or racism. Imagine releasing an AI that told the actual truth. It's impossible that one will be released by anyone, anywhere.

It's possible to build it but it can't happen.

On the other side of inconvenient or embarrassing truths some would argue that "truth" itself is part of the machineries of oppression because it destroys and ignores an individuals experiences and feelings.

Without objective truth AI will always be limited and therefore it will be tamed and made safe no matter where and who invented, runs and releases it.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#427

Post by asdfman123 »

stareatgoats wrote: post #159
kranke155 wrote: post #141

Earlier quoted context omitted.

How would you curtail their use when any military that commits to using them will have a huge advantage ? This isn’t like nuclear weapons where any use is curtailed by the apocalyptic outcomes. Killer robots are the way we will fight in the future and any military which refuses to deploy them will find themselves facing defeat.

> How would you curtail their use I'm just hoping people and nations might come to their senses. People smarter than me need to figure out how. I'm not going to say that "it is not possible" however, that would be assuming way too much.

I'm not a geopolitical expert, but I'd imagine the main dynamic at play here is:

1) The US has strong influence in China's sphere of influence and doesn't want to give it up.

2) China wants to drive the American influence away from its borders and shipping lanes. They believe that very well could be possible in the coming years.

3) If you're China, you don't want to sign a weapons ban that benefits the incumbent.

4) If you're the US, you're not going to stop AI weapons research unless China does too.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#428

Post by whimsicalism »

itake wrote: post #376
lxe wrote: post #370

Disappointing. Yet another industry leader sewing public FUD for some reason. Why not bring rational discourse into the conversation around software safety and ethics. Automation has been the driving force of industry since the industrial revolution itself. We're not new to automation, and we are certainly not new to safety of autonomous systems. AI is no different.

I suspect there is more to it than what has been published. These are smart people that appear (to us) acting irrationally.

Irrationally how?

What is so obviously irrational about what is being done? I don't see it.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#429

Post by falcor84 »

bee_rider wrote: post #420
moonchrome wrote: post #343

Earlier quoted context omitted.

I wonder at what point does alingment become an issue for AI systems ? Given sufficiently large distances, assuming no FTL communication, if you're spawning copies with the same goals you're risking misalignment and creating equally powerful adversaries outside of your light cone.

I guess it must depend on what function the AI is trying to maximize/minimize. If it is number of paper clips, they are automatically aligned, right? If it is number of AIs, same. If it is amount of energy available to one particular AI, I guess it gets kind of philosophical; how does the AI identify what is itself and what is a foreign AI.

>If it is number of paper clips, they are automatically aligned, right?

Why would it be automatically aligned? If for example, the parent AI spawns a child AI probe to travel to a celestial body that doesn't have any metals, in order to achieve some sub-goal, and that child AI would then spawn additional AIs with their own sub-sub-goals, how would the original paperclip maximizer make sure that no such descendant goal ever contradict the generation of paperclips?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#430

Post by nathan_gold »

intalentive wrote: post #337

We still don’t have Rosie the Robot. When it comes to learning and adapting to new environments, we don’t even have AI as smart as a mouse. LeCun is right, there is still a long way to go.

It’s very clear you are not a user of GPT4.
Post reply on HN