Machine intelligence, part 2
blog.samaltman.com
Machine intelligence, part 2
1–10 of 157 posts
Re: Machine intelligence, part 2
#2Thanks for writing. We as a species need to talk more about this.
What's the goal in requiring the first SMI to detect other SMI being developed but take no action beyond detection? Or did you mean to say that it should detect other SMI under development and then report those projects to whatever regulatory authority is overseeing SMI?
Also, perhaps a better framework for a "friendly" SMI than Asimov’s zeroeth law is the "Coherent Extrapolated Volition" model (http://en.wikipedia.org/wiki/Friendly_artificial_intelligenc...). Nick Bostrom and many others believe this may be the most workable safeguard.
Re: Machine intelligence, part 2
#3"The US government, and all other governments, should regulate the development of SMI"
This has basically the highest insanity * prominence of speaker product I've ever seen. The same geniuses who want to ban encryption and can't manage to get a CRUD website up and running without a 8-figure budget will regulate the existence of algorithms (i.e., math) routinely generated by groups of Of course, people will be glad to "demonstrate their capabilities" to the group that's responsible for shutting them down if they're "too capable".
"it’s important to write the regulations in such a way that they provide protection while producing minimal drag on innovation (though there will be some unavoidable cost)."
Has this man ever seen a government regulatory framework operate?
"To state the obvious, one of the biggest challenges is that the US has broken all trust with the tech community over the past couple of years. We’d need a new agency to do this."
Don't worry guys! This is a new agency!
"Regulation would have an effect on SMI development via financing—most venture firms and large technology companies don’t want to break major laws. Most venture-backed startups and large companies would presumably comply with the regulations."
Here's the nugget. The only way this article makes any sense at all is as a completely disingenuous argument devoted to making it impossible to invest in technology companies, period (please tell me what "technology company" does not have at least some algorithmic or optimization component), without having a large compliance department and having greased the right palms. I don't particularly think he's that Machiavellian, but it seems like that or the alternative...
Re: Machine intelligence, part 2
#4I immediately thought of the Turing Police in Neuromancer when I read this post.
Re: Machine intelligence, part 2
#5Remember when the U.S. tried to limit the sale of PS3s to the government of Iran? That was a decade ago. In light of current negotiations with the Islamic Republic I see a lot of parallels between nuclear non-proliferation and the regulation of "weaponized" AI. To wit, even if the entire International System is in agreement about a set of regulations that will do little to stop bad actors. And do I really want inspectors coming to my office to poke around my jump point search code ;)
Rather, let's take a cue from Nassim Nicholas Taleb and in concert with the progression of AI, develop counter systems that make us less fragile. It shouldn't be unreasonable to extrapolate where AI is headed in 1, 5 or even 20 years based upon publicly available research.
Which brings up the most crucial point. The subset of humanity that constitutes world-class AI researchers is very finite indeed. Which brings up the significant possibility of a Bane / Dr. Pavel scenario playing out. Not unlike certain coerced atomic scientists in Persia today.
Re: Machine intelligence, part 2
#6"The US government, and all other governments, should regulate the development of SMI" This has basically the highest insanity * prominence of speaker product I've ever seen. The same geniuses who want to ban encryption and can't manage to get a CRUD website up and running without a 8-figure budget will regulate the existence of algorithms (i.e., math) routinely generated by groups of Of course, people will be glad t…
What I took from that line is that we need a regulatory power and that our current best option is the US government. SMI without regulation is impermissible. I don't like opening the regulation from within the U.S. Government, but I dislike other options more.
Re: Machine intelligence, part 2
#7"The US government, and all other governments, should regulate the development of SMI" This has basically the highest insanity * prominence of speaker product I've ever seen. The same geniuses who want to ban encryption and can't manage to get a CRUD website up and running without a 8-figure budget will regulate the existence of algorithms (i.e., math) routinely generated by groups of Of course, people will be glad t…
We could call it the Turing Police...
Re: Machine intelligence, part 2
#8Let's take a good look at what happens when American companies try to push for some incredible goal without stopping to think: See the Apollo missions- Assuming creating the next step in our spiritual evolution is a good idea, we shouldn't let this hype push us harder than we are supposed to go. I agree that we need to slow down and be very careful. However regulation is not the complete answer. We need to completely restructure our society before we introduce the tool of tools, to ensure that we can constantly use it with love and empathy in mind, rather than the blatant and current misuse of high-tech for greed, sloth, and wrath.
Re: Machine intelligence, part 2
#9Why is this all of a sudden such a hot topic? We're nowhere near anything resembling general intelligence even at the level of a toddler. Barring any kind of hardware innovations we will not get there any time soon either with just software.
Re: Machine intelligence, part 2
#10On the one hand we have lay people (including Altman and Musk) who are warning that AI is progressing too fast. On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter[1].
The crux of the problem is that AI experts largely do not see meaningful progress on any axis that makes an AI doomsday scenario plausible, which causes staunch disbelief and ridicule of people who write things like this.
So realistically if we want to resolve this issue, the place to start is not public policy. That's actually part of the problem -- tech policy is bad largely because it is mostly legislated by people who don't understand the technology. We have to start with a discussion about how this progress towards doomsday AI should be measured, and then formulate policy to directly address those scenarios. As long as Altman et al are extrapolating on axes I (for one) don't understand, I honestly think this conversation is unlikely to be productive.
[1] Most notably, a similar hurrah happened when neural networks first became popular. There were some promising early results, which led to much enthusiasm, and then Papert and Minsky published "Perceptron", whose Group Invariance Theorem cast aspersions on whether simple neural networks were really doing what we thought they were. Eventually we found out that many early experiments just had a bad methodology, which very nearly stopped research interest in the field for almost a decade. Now we are in a period of AI hype again, and when you hear people like Andrew Ng get up in front of audiences and say things like "for the first time in my adult life I am hopeful about real AI", you might start to wonder whether there is a lesson here.
EDIT: Friends, I am confused by the downvotes. :) I am happy to have a discussion about this topic!