Superintelligence: The Idea That Eats Smart People • Hacker News Board

Re: Superintelligence: The Idea That Eats Smart People

#601

Post by Nevermark » Sun, Dec 25, 2016, 7:23 AM UTC

ebspelman wrote: post #362

kobayashi wrote: post #279

Earlier quoted context omitted.

Too late to edit, so I'll post just a few examples here: >The only way out of this mess is to design a moral fixed point, so that even through thousands and thousands of cycles of self-improvement the AI's value system remains stable, and its values are things like 'help people', 'don't kill anybody', 'listen to what people want'. Bostrom absolutely did not say that the only way to inhibit a cataclysmic future for hu…

>> "A machine intelligence need not share any of that complexity" I think this is an assumption that both Bostrom and yourself would acknowledge as such. We simply don't know enough about the nuances of intelligence to make an assertion that there are inherently varying kinds of it.

Well we know there are different kinds in the animal kingdom.

The octopus brain developed completely independently from our brain for instance. The octopus does not learn socially, is able to function independently from birth, and learns everything it does in a few years, as its lifespan is short.

So there must be some major differences due to very different origin and learning styles.

Re: Superintelligence: The Idea That Eats Smart People

#602

Post by nickbauman » Mon, Dec 26, 2016, 3:58 AM UTC

joefourier wrote: post #456

nickbauman wrote: post #83

Moore's Law (on a single CPU core) effectively ended around 2009. Our programming idioms for parallel processing are still so crude, hardly anyone can write maintainable parallelized code for anything but toy problems. Until we can restart Moore's Law again, this is all academic. http://www.globalnerdy.com/2007/09/07/multicore-musings/

Toy problems? GPGPU is widely used in real-world applications such as image processing, compression, computer graphics, physical simulations, cryptography and machine learning, which are all embarrassingly parallel. With regards to programming, I would not all CUDA and OpenCL crude at all.

Interesting in that almost all those problems are share-nothing, i.e. no shared state.

Re: Superintelligence: The Idea That Eats Smart People

#603

Post by hetid » Mon, Dec 26, 2016, 8:07 AM UTC

nojvek wrote: post #539

grkvlt wrote: post #165

Earlier quoted context omitted.

But the law says nothing about cores or clock speeds etc. just transistors and their sizes. The regular reductions in feature size on dies are still happening, as I understand it.

Size of copper atom is 0.2nm. We are already at 10nm fab processes. We can't really get too small otherwise it gets unstable and heat becomes a major issue. On the positive side, electric flows much faster than synapses and transistors are smaller than neurons. So with today's technology if we were to pack silicon parallel cores in size of human brain we'd have a much faster brain.

The heat issue makes the latter impossible.

Re: Superintelligence: The Idea That Eats Smart People

#604

Post by alouisos » Mon, Dec 26, 2016, 4:22 PM UTC

I would love to see more articles like that from people building our current AI to discuss real problems on current AI systems so we can start working on solutions. For example human level interpretability as required by law in EU for AI systems eg targetted ads is a limiting factor to AI progresss because some of the most advanced currents AIs are not interpretable and maybe shouldn't because we can now engineer intelligence different from ours but not necessarily dangerous (or work on making it safe). To my opinion this is a more pressing matter than a divine future paper clip AI killer. Making an assumption of a so called "self recursive superAI" and taking it from there is actually diminishing the power of arguments towards the dangers of AI which is an important discussion sometimes abused by people that have never actually built one and extrapolate Gant mind philosophy arguments towards a dangerous future, which is impressive but avoids any proposal of solution to current potential AI dangers which should maybe included as part of the arguments. Important matters as AI safety can be collectively discussed by engineers and philosophers together based on current state and near future potential and not only as a sci-fi future of a god like entity that has nothing to do with our current AI situation. My two cents, hope I am not offending anyone.

Re: Superintelligence: The Idea That Eats Smart People

#605

Post by Confusion » Tue, Dec 27, 2016, 11:36 AM UTC

Nevermark wrote: post #596

Confusion wrote: post #435

Earlier quoted context omitted.

It's not an ad hominem argument if the personal characteristics are relevant to the topic being discussed. The personal characteristics of the people in his example are have empirically been found to be a good indicator of crankhood.

Um, no. Hawking, Musk, et. al. are highly successful people with objectively valuable contributions who are known to be about to think deeply and solve problems that others have not. They are as far from cranks as anyone could possibly be. Anyone can find non-argument related reasons to suggest anyone else is crazy or a cultist, because no human is completely sane. What someone cannot do (credibly), is claim that rea…

Those were not the people used as an example. The example was real cultists, with robes and rituals that make no sense. And it's not an ad hominem to dismiss them outright based on their appearance. If someone I clearly perceive to be a cultist walks up to me in a mall, I'm not interested in hearing what they have to say, because the odds are empirically large that they will be wasting my time.

Re: Superintelligence: The Idea That Eats Smart People

#606

Post by jodrellblank » Fri, Dec 30, 2016, 2:57 AM UTC

trishume wrote: post #215

tlb wrote: post #38

Earlier quoted context omitted.

Certainly the assumption that every intelligent agent will want to recursively self-improve is unwarranted. But it only takes one intelligent agent that wants to self-improve for the scary thing to happen.

Why wouldn't it if it is able to? It doesn't have to "want" to self-improve, it only has to want anything that it could do better if it was smarter. All it needs is the ability, the lack of an overwhelming reason not to, and a basic architecture of optimizing towards a goal. If you knew an asteroid would hit the earth 1 year from now, and you had the ability to push a button and become 100,000x smarter, I would hope…

Not without knowing more.

Would I get a 100,000x bigger head, and die of a snapped neck? Would I get a 100,000x increase in daily calorie requirements and die within a day? Would I get a 100,000x increase in heat output in my head and have my brain cook itself? Would I get 100,000x more neurons but untrained, so I need to live 100,000x more lifetimes for them to learn something? Would I have 100,000x more intelligence but bottlenecked by one pair of eyes, ears, vocal chords, arms and legs so it's no more applicable?

A 100,000x more intelligent me only represents ~1/50,000th of the world's combined brainpower (assuming 5 billion reasonably capable adults) - why would I think myself to have the best chance of saving the world compared to "everyone working together", if a 100,000x boost to one person represents a fraction of a fraction of a percent of a change to world brainpower?

Unless you're just handwaving hoping for a techno-magic wormhole style fix, we already know what kind of things we'd need to stop an asteroid destroying the world - nukes on rockets, mass evacuation of ground zero areas, maybe high powered lasers, underground survival bunkers - generally things which take large amounts of teamwork and resources way beyond a single person's abilities to build.

It's not clear cut that the button would be automatically a good thing to press, especially when there's no talk of trade-off's or compromises. If you had to get to work in 1 minute instead of 1 hour, would you press a button that made your car 100,000x faster? No, because that would be completely uncontrollable and you'd die as soon as your car hit the first corner and you didn't steer fast enough and hit a building at 1Mmph.

Re: Superintelligence: The Idea That Eats Smart People

#607

Post by brw12 » Wed, Jan 04, 2017, 11:04 PM UTC

So many people don't realize that as social creatures, we see the world through the lens of our social brains. Computers don't care about "all powerful" or "savant" or "should". They don't qualitatively distinguish between killing one person and every person. Self-replication plus proliferation of cheap components plus proliferation of AI algorithms equals a time when a script kiddie or a bug can mean every last fragile sack of meat and water gets punctured or irradiated or whatever. If it can be done in a video game with good physics simulation, it can be done eventually in real life. It won't be like a movie where the ticking time bomb works on a human timescale and always has a humanlike weakness. Comparing this to nuclear weapons is silly. It's more like issuing every person in the world a "kill or help between 0 and 7 billion people" button that's glitchy and spinning up 1,000 4chan chains with advice on tinkering with it.