Superintelligence: The Idea That Eats Smart People • Hacker News Board

Re: Superintelligence: The Idea That Eats Smart People

#481

Post by lmm » Fri, Dec 23, 2016, 1:23 PM UTC

idlewords wrote: post #56

apsec112 wrote: post #29

"Not many people know that Einstein was a burly, muscular fellow. But if Einstein tried to get a cat in a carrier, and the cat didn't want to go, you know what would happen to Einstein. He would have to resort to a brute-force solution that has nothing to do with intelligence, and in that matchup the cat could do pretty well for itself." This seems, actually, like a perfect argument going in the other direction. Ever…

I'm not arguing that it's useless to outsmart a cat. I'm disputing the assumption that being vastly smarter means your opponent is hopelessly outmatched and at your mercy. If you're the first human on an island full of tigers, you're not going to end up as the Tiger King.

And yet somehow humans rule the planet and tigers are an endangered species, surviving only as a result of specific human efforts to conserve them because some humans care about doing so.

How well an AI could survive on a desert island is an irrelevant question when Amazon, Google and dozens of others are already running fully (or as near as makes no difference) automated datacentres controlled by a codebase that still has parts written in C. Hawking can easily get the cat in the container: all he has to do is submit a job to taskrabbit.

Re: Superintelligence: The Idea That Eats Smart People

#482

Post by lmm » Fri, Dec 23, 2016, 1:27 PM UTC

SerLava wrote: post #60

md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

I don't understand why people have such a weird problem reconciling the brain with the mind. It IS all physical. It's also an unimaginably complex fucking shitload of tiny physical objects working incredibly quickly. If your brain was big enough to see the parts working like little gears of a clock, it would probably be planet sized, or something like that. Huge. I would EVEN say it doesn't raise any interesting phil…

> Semi-related, if you made a giant brainlike calculator out of water pipes and valves and shit, and asked it a question, then turned all its pipes and valves for a while... it would probably just say "please kill me."

Um why? Most humans don't do that.

Re: Superintelligence: The Idea That Eats Smart People

#483

Post by krapp » Fri, Dec 23, 2016, 1:29 PM UTC

ccozan wrote: post #460

kabdib wrote: post #445

Earlier quoted context omitted.

You don't even need to do that; just stop mining coal, or operating oil and gas fields, or scram the reactors. Or more easily, open some circuit breakers; in a pinch you can take down a few electrical towers (not many). A rogue AI's "oxygen" is electrical power, which is really kind of fragile. The emergency power for most datacenters won't last more than a few days without replenishing diesel fuel. For a distributed…

What stopps the AI to take the path from Matrix?

If you mean, what stops an AI from locking humanity into a virtual reality simulation and using its collective body heat as fuel - simple thermodynamics.

The Matrix of the original film was originally envisioned as a bootstrap AI, in which the machines were farming humans for their collective processing power - and the simulation that was the Matrix was integral to this purpose, as it served as an operating system for the imprisoned human minds. However, the film's backers felt that concept was too complex for the average moviegoer, and forced the change to "humans = batteries."

But, the canonical Matrix would be too inefficient to actually work. The amount of energy needed to maintain a human being over the course of an entire lifetime far surpasses the amount of energy that can be harvested as body heat - and adding some kind VR simulation over that just to keep people who are already physically trapped from "escaping" would just a pointless waste of resources.

Re: Superintelligence: The Idea That Eats Smart People

#484

Post by narag » Fri, Dec 23, 2016, 1:31 PM UTC

seiferteric wrote: post #65

md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

One thing I rarely see discussed is the possibility that the brain is just too complex to practically reproduce. That is to say, that it is technically possible, but not practical. Evolution had billions of years, working in a massively parallel way to work this out after all. It's possible that the brain is a huge tangled mess of rules and special cases that we will never be able to fully understand and reproduce. A…

It's possible that the brain is a huge tangled mess of rules and special cases that we will never be able to fully understand and reproduce.

It doesn't seem possible. Our brain, like the rest of our bodies, develops from a single cell. It's DNA (maybe also some other markers) that directs the differentiation of every part. So DNA is where the blueprints of that huge mess would be.

I believe that self-organization is a better way to explain how mind emerges. There's a basic try and error mechanism with strong rewards. The rest is the result of a life of learning.

Re: Superintelligence: The Idea That Eats Smart People

#485

Post by falcolas » Fri, Dec 23, 2016, 1:34 PM UTC

nemof wrote: post #414

coldtea wrote: post #395

Earlier quoted context omitted.

> The problem arises when non-state actors wreak havoc with drones and AI with inpunity. Well, for the rest of the world, state actors wreaking havoc with drones and AI with inpunity is already a problem.

truth. how we collectively became accepting of drones used by our governments to destroy targets half way round the world whilst the pilots sit in some skyscraper somewhere in our own countries is remarkable. honestly the disconnect is beyond deeply troubling.

There is no disconnect for those drone pilots, who become as troubled as anyone who killed someone with a lesser distance between.

Re: Superintelligence: The Idea That Eats Smart People

#486

Post by Sean1708 » Fri, Dec 23, 2016, 1:39 PM UTC

wyager wrote: post #352

AaronFriel wrote: post #346

Earlier quoted context omitted.

I think the current state of deep neural networking designs and research funding pouring into generalizing the simple neural nets we're working with now suggests that we are, in fact, pulling them out of a hat. Right now we're just discarding all the ones that are defective, at a stupendously high rate as we train neural nets. I can't speak to what method would generate the first strong AI, but I suspect the overall…

That's not how neural network training works. It's not magic or guesswork; it's essentially glorified curve fitting (over a much more complicated space than the polynomials). It's also not random in any respect. The space of all neural networks plausibly generated from a training set is very, very small compared to the space of all networks of that size.

I get the feeling that not many people in this thread actually know how AI and related concepts work.

Re: Superintelligence: The Idea That Eats Smart People

#487

Post by falcolas » Fri, Dec 23, 2016, 1:39 PM UTC

kabdib wrote: post #445

Senji wrote: post #384

Earlier quoted context omitted.

You can always use LEO EMP nukes to bring us back to the stone age, thus taking out the AI.

You don't even need to do that; just stop mining coal, or operating oil and gas fields, or scram the reactors. Or more easily, open some circuit breakers; in a pinch you can take down a few electrical towers (not many). A rogue AI's "oxygen" is electrical power, which is really kind of fragile. The emergency power for most datacenters won't last more than a few days without replenishing diesel fuel. For a distributed…

With our current push towards solar and wireless, don't you think these particular circuit breaker paths against a rogue AI are going to be unavailable sooner than later?

Antennas and solar panels can be smashed, but they can also be protected, since they would be mostly concentrated in one area.

Throw in EMP hardened circuitry, and things get a bit harder to destroy.

Re: Superintelligence: The Idea That Eats Smart People

#488

Post by VexXtreme » Fri, Dec 23, 2016, 1:43 PM UTC

fergie wrote: post #443

simonh wrote: post #397

Earlier quoted context omitted.

No, because their power is limited by their role in society and the organization in which they operate. Even the president of the United States has limited powers. If Obama just decided one day that the best course of action was to nuke Moscow and doubled down on doing so it's extremely unlikely he'd be able to do so. There are enough other people with careers, jobs, pension plans and common decency between him and a…

This is a really strong argument for big government. The smaller our all-powerful military command is, the easier it is for it to go rogue.

A counterargument would be that it's harder for a big government to change direction easily, once decided and set in motion. Wrong decisions can compound because it's easier for a large organization to stay its course, even in the face of increasing harm, as evident by the Vietnam war.

Re: Superintelligence: The Idea That Eats Smart People

#489

Post by eli_gottlieb » Fri, Dec 23, 2016, 1:46 PM UTC

lmm wrote: post #477

tboyd47 wrote: post #475

Earlier quoted context omitted.

No doubt that his "inside arguments" have been rebutted extensively by the strong AI optimists and their singularity priests. After all, dreaming up scenarios in which robotic superintelligence dominates humanity is their version of saving the world. That's why I found the "outside arguments" here equally important and compelling. > The outside view doesn't care about content, it sees the form and the context, and it…

> If it sounds like and acts like a cult, why should we treat it any differently from a cult? Even if the people in it are all very smart, wealthy, well-dressed, and appear very rational, they're still preaching the end of the world on a certain date. All of those groups have only one thing in common: they're all wrong. Occasionally the crazies are right. Remember when the idea that the NSA was recording everyone's e…

>Remember when the idea that the NSA was recording everyone's emails was paranoid conspiracy theory talk

No, I remember the time that we'd known there were various signal intelligence programs operated by the US government for decades, and it was just a matter of guessing what the next and biggest one would be.

Re: Superintelligence: The Idea That Eats Smart People

#490

Post by sorokod » Fri, Dec 23, 2016, 1:46 PM UTC

otakucode wrote: post #416

modeless wrote: post #74

Earlier quoted context omitted.

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Human beings have been extremely easy to kill for our entire existence. No system of laws can possibly keep you alive if your neighbors are willing to kill you, and nothing can make them actually unable to kill you. Your neighbor could walk over and put a blade in your jugular, you're dead. They could drive into you at 15MPH with their car, you're dead. They could set your house on fire while you're asleep, you're de…

The difference is that AI might have an "intent". It may be just a statistical contraption married to some descendent of a heat seeking missile sensor, but from the outside it will look like "intent". Perhaps even without the double quotes.