People have a series of rationalizations. People say for example that science
and technology have their own logic, that they are in fact autonomous. This
particular rationalization is profoundly false. It is not true that science
marches on in defiance of human will, independent of human will, that just is
not the case. But it is comfortable, as I said: it leads to the position that
"if I don't do it, someone else will."
Of course if one takes that as an ethical principle then obviously it can serve
as a license to do anything at all. "People will be murdered; if I don't do it,
someone else will." "Women will be raped; if I don't do it, someone else will."
That is just a license for violence.
Other people say, and I think this is a widely used rationalization, that
fundamentally the tools we work on are "mere" tools; This means that whether
they get use for good or evil depends on the person who ultimately buys them
and so on.
There's nothing bad about working in computer vision, for example. Computer
vision may very well some day be used to heal people who would otherwise die.
Of course, it could also be used to guide missiles, cruise missiles for
example, to their destination, and all that. You see, the technology itself is
neutral and value-free and it just depends how one uses it. And besides --
consistent with that -- we can't know, we scientists cannot know how it is
going to be used. So therefore we have no responsibility.
Well, that is false. It is true that a computer, for example, can be used for
good or evil. It is true that a helicopter can be used as a gunship and it can
also be used to rescue people from a mountain pass. And if the question arises
of how a specific device is going to be used, in what I call an abstract ideal
society, then one might very well say one cannot know.
But we live in a concrete society, [and] with concrete social and historical
circumstances and political realities in this society, it is perfectly obvious
that when something like a computer is invented, then it is going to be adopted
will be for military purposes. It follows from the concrete realities in which
we live, it does not follow from pure logic. But we're not living in an
abstract society, we're living in the society in which we in fact live.
If you look at the enormous fruits of human genius that mankind has developed
in the last 50 years, atomic energy and rocketry and flying to the moon and
coherent light, and it goes on and on and on -- and then it turns out that
every one of these triumphs is used primarily in military terms. So it is not
reasonable for a scientist or technologist to insist that he or she does not
know -- or cannot know -- how it is going to be used.
-- Joseph Weizenbaum