Machine intelligence, part 1

So do you take his thoughts seriously or not? Do you now think AI researchers are engaging in unethical behavior since they don't care about AI safety?

The main problem is that most of the time arguments about humanity obviating itself are couched in a framework where the only thing that has advanced, in this instance artificial intelligence, is the science needed to make it a reality. This has never been the case, which is why you see so much eye rolling when arguments like this (or some of the others here about robots replacing human workforces) are made.

How can we say that by the time we have such wondrous machines that we as a species will not have found ways to move ourselves forward to a place on equal footing with whatever we create? Why do we assume that humanity won't move past our current societal constructs when we introduce new actors into the mix? These are the questions we should be asking when someone writes or speaks about the perceived dangers of some future event.

In light of this, while some of the dissent may seem opinionated, I would argue that the original premise of the article is somewhat opinionated itself. I think it goes without saying that most of us would prefer that humanity not obviate itself - but when we think about it do we really believe that the technology to create hyper intelligent machines will come before our society adapts to handle them? The answer may be yes, but lets not pretend such technology will be born into a world that looks like today.

I think people might overestimate the effect of machines being capable of improving themselves.

I don't believe in "magic algorithm"s, that we can't find, that perform dramatically better, but computer can find. I think we are capable of taking, in a few years time, advantage of most hardware. Dramatic improvements in performance thus require improving physical matrix the computation happens in. That means the prospective machine intelligence needs to have access to all the infrastructure thats needed to design and produce such matrix. It also will need to make experiments, and do things by trial-and-error, just like we do, even tho it could catch a hint easier than we do.

Think of chess. Seeing a couple of moves further in to the game might require 1000 times more raw computation. Such power might guarantee victory over lesser equipped competition. It still doesn't mean you'd be even close to "solving" the game. What I mean is, "intelligence" is not the only requirement needed for self-improvement. In the end, it might not even be the bottleneck at all. For example, suppose we are planning the next generation of a transistor for some IC. Suppose a team of humans needs to do 100 experiments to get the process right. A perfectly smart AI might still need to run 90 experiments, no matter what, because it still needs to extract the same information that humans would from those experiments.

The machine needs to do all the chores (of R&D) that we already are doing. I'm not saying they couldn't be done better, but I'm saying they still need to be done, and any creature doing them still needs to spend the time and resources needed to do them. And it's not all brainwork. I have no doubts vanilla humans can be outcompeted at some point. But even when human performance has been exceeded, technology might not immediately take off rocket-like to a world we cannot comprehend.

Still, I wonder, what the hell are we all going to do when paying the power-bills and capital costs of AI become cheaper than hiring a human..

I have an experience to share with you: I taught English a a car factory for a year once, and the director of the paint shop was one of my students. He was describing the process for painting a car, and telling me that the hardest part was painting the inside, as you had to open the doors so that the robot paint head could get inside and do its job. So the body rolls into position, 2 guys open the doors, the robot painter goes in and does its thing, comes out and the guys close the doors again. Rinse and repeat for 20 hours a day.

I asked him why there isn't a robot to open and close the doors. He said that there is, but in this country its much cheaper to pay people to do it. (I think he said the cost of maintaining the door opening robots to be about $1m a year approximately.

So until we have a world where every country has the same salaries as USA/Western Europe/Japan, I think you will always be able to find work.

Even if that work should really be done by a robot

I'd like to see that as well. Machine intelligence is certainly an existential threat, but on the other hand, it's also one of the single largest improvements we could possibly create (insofar as it'd be the last we'd ever need to).