Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We could stumble into AI catastrophe

cold-takes.com

Post reply on HN
Print view View original

41–50 of 122 posts

Re: We could stumble into AI catastrophe

#41

Post by gala8y »

Animats wrote: post #30
dwohnitmok wrote: post #22

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. We have that now. It's called a corporation. See yesterday's article about Exxon.

Which one?

Re: We could stumble into AI catastrophe

#43

Post by JumpCrisscross »

Animats wrote: post #30
dwohnitmok wrote: post #22

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. We have that now. It's called a corporation. See yesterday's article about Exxon.

> called a corporation

That’s banal, self-interested human evil. Corporations and governments alike have perpetuated mass crimes and cover-ups.

Re: We could stumble into AI catastrophe

#45

Post by JumpCrisscross »

mcculley wrote: post #20
mattkrause wrote: post #13

Earlier quoted context omitted.

No, they absolutely will---and already do. The "moral" buck doesn't stop with the algorithm though. Whoever built, configured, or authorized the system is ultimately responsible. By analogy, my oven controls its heating element, but if dinner gets burnt, that's on me. That shouldn't change just because the control policy is more complicated.

We already have a mechanism for absolving responsibility: the corporation. It allows shareholders to profit and not lose more than invested. When they invest in tobacco companies, for example, the system is working as designed.

> allows shareholders to profit and not lose more than invested

To a degree. (See: piercing the veil.) Also, corporations need an authorised signer, who under current law must be a natural person.

Re: We could stumble into AI catastrophe

#46

Post by dustbitying »

Animats wrote: post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

> Reduced need for education. Right now, about half of college graduates do jobs that don't require a college education. For many people, going to college is not cost-effective. That will increase.

you seem to implicitly assume that a college education is an unnecessary expense for most and that therefore, it shouldn't be really given away so readily. this suggests to me that you assume that the real purpose of college is job preparation. your reasoning makes sense from the perspective of a higher level institution (or corporation, or possibly a government) seeking to be as efficient as possible regardless of the impact on typical human individual's well-being.

college is not a 'factory' (or any sort of industry) that 'manufactures' workers for companies.

Re: We could stumble into AI catastrophe

#47

Post by hn_throwaway_99 »

Animats wrote: post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

I think this is a great post, but it also makes me so sad because it highlights a lot of the general pessimism I've felt about the direction of tech over the past decade or so (after so much optimism in the 90s and 00s). I mean, I feel like your description of "Really effective targeted advertising" is spot on, but it also makes me think "Why don't they just dump as all in the nutrient baths Matrix-style and just hook our eyeballs up to a continuous loop of ads?"

In seriousness, I feel like the only hope here is that the general population gets more and more aware about the negative impacts on all this hyper-targeted advertising. Certainly we've been making strides there with things like GDPR and documentaries like the Social Dilemma, but with so much money on the line it will be hard to fight back.

Re: We could stumble into AI catastrophe

#48

Post by vletal »

I agree with many of the concerns outlined in the post. On the other hand the humans depicted in the example dialog behave so dumb they could come from a C level soup opera. If I was a bit more close minded I can see myself stopping listening at that point. Undermines the message.

Re: We could stumble into AI catastrophe

#49

Post by hn_throwaway_99 »

dustbitying wrote: post #46
Animats wrote: post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

> Reduced need for education. Right now, about half of college graduates do jobs that don't require a college education. For many people, going to college is not cost-effective. That will increase. you seem to implicitly assume that a college education is an unnecessary expense for most and that therefore, it shouldn't be really given away so readily. this suggests to me that you assume that the real purpose of colle…

These days, if you want to just "learn for learning's sake", or for self improvement, there are tons of free and cheap ways to get a good education.

The only reason people are willing to pay tens of thousands of dollars for college is that it is a credentialed degree that people think will open up career doors. We shouldn't beat around the bush on this fact.

Re: We could stumble into AI catastrophe

#50

Post by quadcore »

dwohnitmok wrote: post #22

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

An important point is that AI does not need to be sentient or conscious to do a lot of damage.

My pet theory is that consciousness could be (not saying I know for sure this is the case but it could as well be) an effect of "survival instinct". That is: "i'll take action to not die".

Now this is very easy to learn from us as we do this all day long so to speak. Most of the way people react on the internet is tinted by the fear of death and survival for example.

What Im saying is that what we intuitively think as very complex could actually be very simple. As simple as some task optimization. Knowing the only thing that could unplug an AI is us, that'd lead to a hell of a fight. And maybe our complete termination.

Post reply on HN