Live data from Hacker News

We could stumble into AI catastrophe

cold-takes.com

51–60 of 122 posts

Re: We could stumble into AI catastrophe

#51
post #8

Earlier quoted context omitted.

You think no company, government, organization, or religion will ever allow algorithms to make choices on how to deploy capital or otherwise affect the world without human approval?

Algorithms deploy capital all the time. Some massive chunk of the stock market is just algorithmic trading, as is pricing on plane tickets, and on marketplaces like ebay and amazon.

Algorithms are easily understood, not blackboxes like much of AI is.

Re: We could stumble into AI catastrophe

#52
post #41
post #30

Earlier quoted context omitted.

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. We have that now. It's called a corporation. See yesterday's article about Exxon.

Which one?

https://news.ycombinator.com/item?id=34358759

Re: We could stumble into AI catastrophe

#53
post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

> Reduced need for education. Right now, about half of college graduates do jobs that don't require a college education. For many people, going to college is not cost-effective. That will increase. you seem to implicitly assume that a college education is an unnecessary expense for most and that therefore, it shouldn't be really given away so readily. this suggests to me that you assume that the real purpose of colle…

We have been framing it that way for a long time though. How many times have you heard someone talk about the average income of high school graduates vs college graduates as a reason to go to college? I agree with you though. Education should be about personal enrichment, not personal “enrichment”.

Re: We could stumble into AI catastrophe

#54
post #30

Earlier quoted context omitted.

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. We have that now. It's called a corporation. See yesterday's article about Exxon.

See also "paperclip maximizer." Then imagine what happens when you have an entire global economy driven by these types of inhuman, amoral agents. https://www.lesswrong.com/tag/paperclip-maximizer

I've also read somewhere about AI feedback loops:

ai generates content ai pushes ads to promote its content.

It's somewhat already happening: recommendation algorithms choose what content we consume, analytics make decisions from this data and make more content that recommendation algorithms like. AI HFT bots can directly influence economy and so on

Re: We could stumble into AI catastrophe

#55
post #29

This author is mostly writing an update of 1950s science fiction about a robot takeover. Things to worry about in the near term: - Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each c…

> - Machines should think, people should work

> - Reduced need for education.

This subset is the radioactive bits of AI impact. We can always address the rest, but that requires thinking and an education.

Re: We could stumble into AI catastrophe

#56
post #30

Earlier quoted context omitted.

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. We have that now. It's called a corporation. See yesterday's article about Exxon.

See also "paperclip maximizer." Then imagine what happens when you have an entire global economy driven by these types of inhuman, amoral agents. https://www.lesswrong.com/tag/paperclip-maximizer

I would say that "billionaire wealth maximization" agents are already giving us a glimpse.

Or rather, they're giving the rest of us a glimpse of what life has been like for those at the "bottom of the pile" for a long time: a world organised around goals essentially completely unrelated and indifferent to those people's existence.

Re: We could stumble into AI catastrophe

#57

Earlier quoted context omitted.

> Reduced need for education. Right now, about half of college graduates do jobs that don't require a college education. For many people, going to college is not cost-effective. That will increase. you seem to implicitly assume that a college education is an unnecessary expense for most and that therefore, it shouldn't be really given away so readily. this suggests to me that you assume that the real purpose of colle…

We have been framing it that way for a long time though. How many times have you heard someone talk about the average income of high school graduates vs college graduates as a reason to go to college? I agree with you though. Education should be about personal enrichment, not personal “enrichment”.

We have been framing it that way for a long time though.

Yes. That changed on February 28, 1967.[1][2]

[1] https://www.chronicle.com/article/the-day-the-purpose-of-col...

[2] https://archive.is/G11YT

Re: We could stumble into AI catastrophe

#58
post #30

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. We have that now. It's called a corporation. See yesterday's article about Exxon.

>We have that now. It's called a corporation.

I think there’s a good argument to say no, we don’t have that now, and these are some ways it could be very different.

Video: Why not just think of AGI as a corporation?

https://youtu.be/L5pUA3LsEawP

Re: We could stumble into AI catastrophe

#59
post #30

Earlier quoted context omitted.

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. We have that now. It's called a corporation. See yesterday's article about Exxon.

> called a corporation That’s banal, self-interested human evil. Corporations and governments alike have perpetuated mass crimes and cover-ups.

Corporations are designed to do so. So spake Milton Friedman.[1] That was when "greed is good, greed works" became respectable.

[1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr...

Re: We could stumble into AI catastrophe

#60

Earlier quoted context omitted.

We already have a mechanism for absolving responsibility: the corporation. It allows shareholders to profit and not lose more than invested. When they invest in tobacco companies, for example, the system is working as designed.

> allows shareholders to profit and not lose more than invested To a degree. (See: piercing the veil.) Also, corporations need an authorised signer, who under current law must be a natural person.

I am not talking about situations where there is really a bad human at the helm. I can buy shares in BP or Altria and not suffer any consequences beyond losing my investment no matter how many people those corporations kill.

Having algorithms and AI models decide how to profit regardless of how it harms humans is the inevitable next step.

Post reply on HN