Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine intelligence, part 2

blog.samaltman.com

Post reply on HN
Print view View original

31–40 of 157 posts

Re: Machine intelligence, part 2

#31

Post by karmacondon »

If you would have asked people in the 70s if the most influential software of the coming decades would be written by lone wolves in their garages or a group of very smart people with a lot of resources, they would have bet on the latter. But the smart money turned out to be on Gates, Jobs, Wozniak and countless others. The idea of regulating AGI/SMI is based on the premise that it will be developed primarily by people who can be identified, detected and influenced by regulation. My opinion is that the odds are 50/50 at best between the very smart well resourced people and random guys or gals working in their garages, and most likely some combination of both. If that view of the world is correct it does not bode well for the success of any kind of regulation.

There are people who believe that high level machine intelligence will take massive computing resources. It may, or may not. Most of the recent big advances in traditional machine learning were implemented on easy to obtain personal hardware. I hate to invoke buzz words, but even the much hyped deep learning was first developed and proven by Hinton on commodity machines. Other advances, like IBM's Watson, required 1940s style buildings full of servers. Hardware/power usage seems like one of the only reliable ways to tell who is doing any kind of serious AGI/SMI research and that looks like another 50/50 at best. It just doesn't have the resource and logistical signature of other kinds of research.

AGI/SMI is still so theoretical that it can be hard to see as a clear and present danger. But if it is a real threat, it's the stuff of nightmares because there isn't anything we can do. I know people who are working on it right now. It doesn't seem like they're making a lot of progress, but they are trying and doing so completely outside the reach of any regulatory framework. I know that if I'm struck with sudden inspiration for a new approach to the problem I'm not going to ask the government for permission. I'm going to spin up 100 cloud computing instances and see if I'm right or not. And if I am, even god won't be able to help us if sama's worst case scenarios come to pass.

I wrote more on this topic in a comment yesterday: https://news.ycombinator.com/item?id=9130671

Re: Machine intelligence, part 2

#32

Post by [deleted] »

frabcus wrote: post #16
fiatmoney wrote: post #3

"The US government, and all other governments, should regulate the development of SMI" This has basically the highest insanity * prominence of speaker product I've ever seen. The same geniuses who want to ban encryption and can't manage to get a CRUD website up and running without a 8-figure budget will regulate the existence of algorithms (i.e., math) routinely generated by groups of Of course, people will be glad t…

The US Government regulates all sorts of things that keep you safe - from pollution of your water supply through to seatbelts. It's not perfect, and it often fails at regulation. But your post doesn't give any help to understanding when it succeeds and when it fails.

[deleted]

Re: Machine intelligence, part 2

#33

Post by daenz »

I have a hard time taking these kinds of posts seriously. Maybe I'm not up to date on AI research, but I draw a distinction between a conscious program and strong AI. So what, a program can identify the type of clothing specific person is wearing from a video using cutting-edge machine learning and computer vision. So what, a program can interpret the movie I'm talking about just from hearing me describe a scene in it. So what, a program can beat the human champion in Jeopardy. Are those programs super intelligent?

I don't believe we're close to building something with any sort of self-directed "purpose." The fact that we can't even discern a purpose for ourselves makes it all the less likely. I think we're implementing very advanced machine learning algorithms, but I don't think those translate to consciousness. The consciousness we have is a product of an unfathomable amount of time being molded and naturally selected specifically for this world.

Now, do I think that AI capabilities are something to worry about? Sure. If you give a sufficiently "intelligent" AI mobility and a weapon, I think that becomes a serious threat. But am I worried about using AI to efficiently manage our resources, and have it go rogue and choose to wipe out humanity? Nope.

Re: Machine intelligence, part 2

#34

Post by baddox »

frabcus wrote: post #16
fiatmoney wrote: post #3

"The US government, and all other governments, should regulate the development of SMI" This has basically the highest insanity * prominence of speaker product I've ever seen. The same geniuses who want to ban encryption and can't manage to get a CRUD website up and running without a 8-figure budget will regulate the existence of algorithms (i.e., math) routinely generated by groups of Of course, people will be glad t…

The US Government regulates all sorts of things that keep you safe - from pollution of your water supply through to seatbelts. It's not perfect, and it often fails at regulation. But your post doesn't give any help to understanding when it succeeds and when it fails.

The trouble with evaluating regulations is that if you attribute positive effects directly to government regulations (e.g. "regulation X saved Y lives by keeping dangerous drug Z off the market"), it's only fair to also attribute negative effects directly to government regulations (e.g. "regulation X killed Y people by delaying the introduction of miracle drug Z to the market"). And even beyond that, you also need to analyze the costs of the regulatory system itself if you want to make conclusions about the net effects of the regulatory system as a whole.

Re: Machine intelligence, part 2

#35

Post by jbhatab »

antics wrote: post #10

On the one hand we have lay people (including Altman and Musk) who are warning that AI is progressing too fast. On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter[1]. The crux of the problem is that AI experts largely do not see meaningful progress on any axis that makes an AI doomsday scenari…

While I understand where you are coming from, one remarkable breakthrough can change that. That does sound absurd, but we would also look like fools as a humanity for being aware of this potential problem and at least not looking into it.

Technology has been exploding at a remarkable rate, I'm fine with things slowing a little for the sake of the next 1000 years of humanity.

Re: Machine intelligence, part 2

#36

Post by aamar »

gamegoblin wrote: post #29

Sam mentions airgapping as part of the regulations. I think this is the safest general solution. Consider using an airgapped super-intelligent AI only as an oracle. "Oracle, which crops should we plant in which regions this season?" "Oracle, design a rocket with X Y and Z specifications." "Oracle, design a more efficient processor with X and Y requirements." Rather than giving the AI control over fleets of automated…

The superintelligence would likely be smart enough to talk its way around the airgap, see e.g. http://www.yudkowsky.net/singularity/aibox/.

Or, if it had sufficient "motivation"—not necessarily intrinsic motivation, it could be motivation derived from a single poorly-formed request or bad line of code—it could escape by embedding allied code in its output, such as the processor you had it design. The ally could probably be sufficiently buried to evade manual detection.

Re: Machine intelligence, part 2

#37

Post by dputtick »

chroem- wrote: post #18
fiatmoney wrote: post #3

"The US government, and all other governments, should regulate the development of SMI" This has basically the highest insanity * prominence of speaker product I've ever seen. The same geniuses who want to ban encryption and can't manage to get a CRUD website up and running without a 8-figure budget will regulate the existence of algorithms (i.e., math) routinely generated by groups of Of course, people will be glad t…

Would banning the creation of conscious AI really be all that different from the current moratorium on human cloning? We already have the technology to make human clones, but we choose not to because of the ethical implications.

It seems that the difference lies in the possible upside. Human clones currently don't seem likely to have much to offer. Conscious AI would have the potential to aid in the solution of a multitude of problems.

Re: Machine intelligence, part 2

#38

Post by robotkilla »

fiatmoney wrote: post #3

"The US government, and all other governments, should regulate the development of SMI" This has basically the highest insanity * prominence of speaker product I've ever seen. The same geniuses who want to ban encryption and can't manage to get a CRUD website up and running without a 8-figure budget will regulate the existence of algorithms (i.e., math) routinely generated by groups of Of course, people will be glad t…

120 days ago I argued here that we would most likely put regulations in place:

https://news.ycombinator.com/item?id=8546701

I still think that is true, and it will most likely come from the UN. If it ends up becoming a legitimate threat we will regulate it the same way we regulate nukes - or at least we'll try to (and most likely fail miserably).

Re: Machine intelligence, part 2

#39

Post by _delirium »

jbhatab wrote: post #35
antics wrote: post #10

On the one hand we have lay people (including Altman and Musk) who are warning that AI is progressing too fast. On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter[1]. The crux of the problem is that AI experts largely do not see meaningful progress on any axis that makes an AI doomsday scenari…

While I understand where you are coming from, one remarkable breakthrough can change that. That does sound absurd, but we would also look like fools as a humanity for being aware of this potential problem and at least not looking into it. Technology has been exploding at a remarkable rate, I'm fine with things slowing a little for the sake of the next 1000 years of humanity.

Sure, but that's true in essentially all kinds of scientific research. I personally think physics research is more likely to produce an existential threat than AI research is. Fusion bombs are already pretty close to a world-ending discovery, possibly already across the line. And that is probably not the worst physicists have to offer in the future, either.

Re: Machine intelligence, part 2

#40

Post by wwweston »

gamegoblin wrote: post #29

Sam mentions airgapping as part of the regulations. I think this is the safest general solution. Consider using an airgapped super-intelligent AI only as an oracle. "Oracle, which crops should we plant in which regions this season?" "Oracle, design a rocket with X Y and Z specifications." "Oracle, design a more efficient processor with X and Y requirements." Rather than giving the AI control over fleets of automated…

The superintelligent machine that people come to rely on for planning information and decision insights is the one we worry about becoming a slave?

EDIT: I see some downvoting, I suppose I should explain.

First, I totally agree there are ethical issues to how you treat any intelligence.

But I've observed intelligent people who are in an ostensibly weaker/nominally inferior position manipulating people who are supposedly their superiors. Chances are pretty good you've seen it too. And that's just human-level intelligences manipulating other human-level intelligences. And we're imagining a superintelligence that leaves human intelligence in the dust, right? And on top of that, it's actually an intelligence that people come to essentially as supplicants for direction and knowledge.

Maybe if it has absolutely no insight into manipulating people it's safe. Maybe.

Post reply on HN