Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine intelligence, part 2

blog.samaltman.com

Post reply on HN
Print view View original

61–70 of 157 posts

Re: Machine intelligence, part 2

#61

Post by msutherl »

lukeprog wrote: post #55
antics wrote: post #10

On the one hand we have lay people (including Altman and Musk) who are warning that AI is progressing too fast. On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter[1]. The crux of the problem is that AI experts largely do not see meaningful progress on any axis that makes an AI doomsday scenari…

> On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter This doesn't seem true. E.g. the signatories on the Future of Life Institute's recent open letter about AI progress and safety include: * Stuart Russell & Peter Norvig, co-authors of the #1 AI textbook * Tom Dietterich, AAAI President * Eric…

That letter is not about "superhuman machine intelligence", it is about "ensuring that AI remains robust and beneficial." Among the threats cited are: autonomous vehicles, machines that need to make ethical decisions, autonomous weapons, surveillance, and issues of verification, validity, and control. Words like "singularity", "superhuman", and "AGI" do not appear.

Re: Machine intelligence, part 2

#62

Post by tolmasky »

gamegoblin wrote: post #29

Sam mentions airgapping as part of the regulations. I think this is the safest general solution. Consider using an airgapped super-intelligent AI only as an oracle. "Oracle, which crops should we plant in which regions this season?" "Oracle, design a rocket with X Y and Z specifications." "Oracle, design a more efficient processor with X and Y requirements." Rather than giving the AI control over fleets of automated…

> 1. Design the cheapest-to-manufacture rocket with a 99.99% success probability of getting to Mars with all crew healthy. A human looks over the designs and confirms they don't involve destroying all humanity.

psh, halting problem.

Re: Machine intelligence, part 2

#63

Post by jal278 »

AceJohnny2 wrote: post #47
daenz wrote: post #33

I have a hard time taking these kinds of posts seriously. Maybe I'm not up to date on AI research, but I draw a distinction between a conscious program and strong AI. So what, a program can identify the type of clothing specific person is wearing from a video using cutting-edge machine learning and computer vision. So what, a program can interpret the movie I'm talking about just from hearing me describe a scene in i…

> So what, a program can identify the type of clothing specific person is wearing from a video using cutting-edge machine learning and computer vision. So what, a program can interpret the movie I'm talking about just from hearing me describe a scene in it. So what, a program can beat the human champion in Jeopardy. Are those programs super intelligent? All these programs are doing great at recognizing things. Neural…

Perhaps you missed this: http://www.nature.com/nature/journal/v518/n7540/full/nature1...

A neural network that learns to play 40 different atari games at human-competitive level given only raw pixels and a few days of play time.

Re: Machine intelligence, part 2

#64

Post by pavedwalden »

lukeprog wrote: post #55
antics wrote: post #10

On the one hand we have lay people (including Altman and Musk) who are warning that AI is progressing too fast. On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter[1]. The crux of the problem is that AI experts largely do not see meaningful progress on any axis that makes an AI doomsday scenari…

> On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter This doesn't seem true. E.g. the signatories on the Future of Life Institute's recent open letter about AI progress and safety include: * Stuart Russell & Peter Norvig, co-authors of the #1 AI textbook * Tom Dietterich, AAAI President * Eric…

The tone of that letter was "Here's a general framework for responsibly building more powerful AI systems", whereas the lay press is reporting "Musk predicts Skynet in 5 years"

http://futureoflife.org/static/data/documents/research_prior...

Re: Machine intelligence, part 2

#65

Post by joeldg »

One issue with SMI and what is known as "The Control Problem" is that you are trying to come up with rules for something that is not just smarter than you, but is 'vastly' smarter than you, and in fact the entire human race put together.

You are basically trying to figure out a way to 'in-effect' cheat a god, which in relation to us, yes that is basically what it would be. An SMI is pretty much the definition of a god (and a possibly crazy greek god as well).

The idea that something like that would turn the universe into paper-clips, is somewhat silly and most of the arguments along these lines discount that "S" part of the whole thing.

What is the answer..

There is none.

Re: Machine intelligence, part 2

#66

Post by api »

fiatmoney wrote: post #3

"The US government, and all other governments, should regulate the development of SMI" This has basically the highest insanity * prominence of speaker product I've ever seen. The same geniuses who want to ban encryption and can't manage to get a CRUD website up and running without a 8-figure budget will regulate the existence of algorithms (i.e., math) routinely generated by groups of Of course, people will be glad t…

"This has basically the highest insanity * prominence of speaker product I've ever seen."

(I edited this post to remove my shock-induced LULZ and other flapdoodle and concentrate on the important stuff.)

The more I re-read this and reflect on it, the more shocked I am. The first post in this serious was a bit of an eyebrow-raiser for me, but it still remained on the sane side of the fence. Yes, dangerous AI is worth thinking about. Lots of things are. Alien invasions aren't completely implausible either, and the Fermi Paradox is definitely worth thinking about. But this has jumped from thinking about the Fermi Paradox to arguing that we halt all radio emissions and space exploration now or the aliens will come for us. This is not a rational leap.

As I said elsewhere in this thread: could this be some kind of ploy to make basic CS research illegal for anyone but deep-pocketed VC-linked concerns (and governments)? Probably not, but that could very well be the result. The sorts of regulations discussed here would make independent CS research illegal. Please tell me this "conspiracy theory" is insane. I really want to believe it is. Someone please tell me I'm crazy. Please? Anyone?

Re: Machine intelligence, part 2

#67

Post by lazaroclapp »

rndn wrote: post #50
antics wrote: post #10

On the one hand we have lay people (including Altman and Musk) who are warning that AI is progressing too fast. On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter[1]. The crux of the problem is that AI experts largely do not see meaningful progress on any axis that makes an AI doomsday scenari…

A lot of experts also deny dangers coming from global warming and other invasions in complex systems we don't fully understand. Scientists in the 1940s also downplayed the danger of the atomic bomb. It's easy to brush away a hypothetical danger with arguments that resources might be wasted or that people in charge might embarrass themselves if their assumptions turn out to be wrong, or that research might suffer due…

By the time the atomic bomb was truly imminent (in the sense of it being within the reach of massive dedicated investment by the most powerful nations on Earth), scientists knew very well of the dangers [1].

As for global warming. No expert denies global warming, and you'll be hard pressed to find many that deny anthropic global warming [2]. This has been the case for many decades.

I am not an AI expert, but from what I can see thus far, our knowledge of cognition seems to be as far from human or post-human intelligence as Nitroglycerin was from the Atomic Bomb (e.g. fundamentally different principles at work, which is not subject to exponential improvement without one or more breakthroughs or qualitatively different approaches). Also, our AI-Nitroglycerin seems scary enough: autonomous weapons (think self-driving self-firing drones, not terminator), massive changes in our means of production that reduce human labor/employment and might cause social earthquakes and the ability to draw meaningful conclusions from surveillance data of 7 billion people...

[1] https://en.wikipedia.org/wiki/Einstein%E2%80%93Szil%C3%A1rd_... (and this seems to have been understood as an approaching risk in the physics community way before anyone thought of telling FDR).

[2] https://en.wikipedia.org/wiki/Global_warming_controversy#Sci...

Re: Machine intelligence, part 2

#68

Post by msutherl »

karmacondon wrote: post #31

If you would have asked people in the 70s if the most influential software of the coming decades would be written by lone wolves in their garages or a group of very smart people with a lot of resources, they would have bet on the latter. But the smart money turned out to be on Gates, Jobs, Wozniak and countless others. The idea of regulating AGI/SMI is based on the premise that it will be developed primarily by peopl…

"If you would have asked people in the 70s if the most influential software of the coming decades would be written by lone wolves in their garages or a group of very smart people with a lot of resources, they would have bet on the latter. But the smart money turned out to be on Gates, Jobs, Wozniak and countless others."

This is a myth. Gates, Jobs, and Wozniak were all smart people (Harvard, Reed, genius) with a lot of resources (access to mainframe as a student, access to Silicon Valley infrastructure and talent, which emerged originally around military contracts and DARPA grants.)

Re: Machine intelligence, part 2

#69

Post by DennisP »

AceJohnny2 wrote: post #47
daenz wrote: post #33

I have a hard time taking these kinds of posts seriously. Maybe I'm not up to date on AI research, but I draw a distinction between a conscious program and strong AI. So what, a program can identify the type of clothing specific person is wearing from a video using cutting-edge machine learning and computer vision. So what, a program can interpret the movie I'm talking about just from hearing me describe a scene in i…

> So what, a program can identify the type of clothing specific person is wearing from a video using cutting-edge machine learning and computer vision. So what, a program can interpret the movie I'm talking about just from hearing me describe a scene in it. So what, a program can beat the human champion in Jeopardy. Are those programs super intelligent? All these programs are doing great at recognizing things. Neural…

Planning is a big part of AI. One example is a recent system[1] that learned to play 49 Atari games, most at better than human skill. It had no specific programming for playing those games, or even video games in general. Its only input was the game video and the score, which it tried to optimize, and it was programmed to learn from its mistakes.

[1] http://www.popularmechanics.com/culture/gaming/a14276/why-th...

Re: Machine intelligence, part 2

#70

Post by ggreer »

dkarapetyan wrote: post #9

Why is this all of a sudden such a hot topic? We're nowhere near anything resembling general intelligence even at the level of a toddler. Barring any kind of hardware innovations we will not get there any time soon either with just software.

> We're nowhere near anything resembling general intelligence even at the level of a toddler. Barring any kind of hardware innovations we will not get there any time soon either with just software.

Should people have discussed the control and proliferation of nuclear weapons decades before their invention? It seems to me that even a century beforehand, the conversation would have been productive. If anything, we got very lucky with nukes. Physics could have easily turned out differently and allowed people to create bombs from substances more common than a few transuranic isotopes.

AI is, potentially, more dangerous than nuclear weapons. If you poll experts, they estimate a 50% chance of human-level AI by 2040-2050.[1] That's 25-35 years away. They also estimate that superintelligence will arrive less than 30 years after that. Lastly, one in three of these AI experts predict that superintelligence will be bad or extremely bad for humanity.

It seems like now is a great time to have this discussion.

1. http://www.nickbostrom.com/papers/survey.pdf

Post reply on HN