AI

I think you are an excellent blogger and glad that you are posting to HN.

That said, I hope that you will think more critically and clearly before publishing vague, fuzzy, uninformed, and unlogical thoughts (not illogical, but unlogical) like the following:

>The biggest question for me is not about artificial intelligence, but instead about artificial consciousness, or creativity, or desire, or whatever you want to call it. I am quite confident that we’ll be able to make computer programs that perform specific complex tasks very well. But how do we make a computer program that decides what it wants to do? How do we make a computer decide to care on its own about learning to drive a car? Or write a novel?

Consciousness, creativity, and desire are all quite distinct things. It is very important for people who are attempting to approach the coming reality of artificial intelligence to be able to distinguish between different things like that.

There have been computer programs that decide what they want to do for decades. Perhaps you were thinking of a specific human-like type of decision process, but if so, you must say so and reason that way. Otherwise you are just conveying some fuzzy thoughts. And the problem is that you are doing so in the context of real scientific undertakings with results directly applicable to your thoughts.

A computer deciding what to care about or learn or what behavior to engage in "on its own" is related to the previous topic you mention, and in and of itself, does not require artificial general intelligence.

How do we make a computer program write a novel? I think that is a good question and an effective answer to it I believe _might_ be in the category of 'real' artificial general intelligence. However, I think that it will probably soon be possible to create 'narrow' AIs that can generate novels without being generally intelligent. http://www.nytimes.com/2011/09/11/business/computer-generate...

The easiest way to create AI would be to make a model our own brain's neural pathways. With a computer model we could analyze and break down what makes consciousness/intelligence, etc.

I want to emphasize just how good this book is. It may very well be my favorite thing I've read and I was also initially skeptical when it was presented as fan fiction.

I think I might restart reading it.

First, the moral argument against inflicting pain on others doesn't depend on existence of pain. The moral dilemma is: is it acceptable to inflict pain on others or not. This is different, and to a large extend independent, from the question if pain exists in experience of others. In other words, if pain exists in others, it doesn't follow that you have to, by mere logical reasoning, make a moral conclusion that inflicting pain is wrong. There is an uncrossable ontological abyss between the empirical what is and the moral what should be.

Second, the case of pain from an empirical side is not at all like what we have in the philosophical zombie "problem". We can empirically observe pain. There are all sorts of physiological and neural manifestations of pain. Of course, now you may say "ah, but how can we know that these empirical manifestations mean the person is experiencing the sensation of pain". Scientifically that dilemma makes little sense, it's simply unproductive, it's scientifically useless. If we were to go by that route, we could inject a similar dilemma into every scientific problem, which inevitably would lead to the problem of solipsism. How can we really, really be certain that anything at all exists? Well, I suppose we can't, but this is a question that science has long ago abandoned because it doesn't get you anywhere, it doesn't yield any useful results.

Do note that unlike the question of pain, the zombie problem is defined so that there is in principle absolutely no way to detect, to measure, if someone is a zombie or not. On the other hand, we can in principle measure and detect events correlated with introspective reports on sensations. If we couldn't do that for some phenomenon it would be wise to consider that the phenomenon doesn't exist for the purpose of empirical scientific examination.

Frankly, I'm surprised that my previous post (where I say the zombie problem is nonsense) got downvoted because this is the foundation of scientific methodology. If you can not, even in principle, measure/detect something then it makes no sense to discuss it. Of course, you can amuse yourself and speculate on it, but that falls outside of boundaries of scientific inquiry and I hope that's what we're discussing here.

> Even if it decides to be our friend and to help our species, someone will of course fork that AI and give it a negative personality and goals.

No: the first AI won't let them. See, we're talking a rapidly improving super-intelligence. Whatever is contrary to its goals, it will squash like a bug. A mad scientist forking the code of the AI with a different goal structure is definitely contrary to the goals of that first super-intelligence, and will be shut down before it grows into a sizeable competitor.

The result of intelligence explosion is a Singleton: the AI will be a perfectly efficient dictator. It may even shield us from the laws of physics until we graduate to adulthood.