Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We could stumble into AI catastrophe

cold-takes.com

Post reply on HN
Print view View original

21–30 of 122 posts

Re: We could stumble into AI catastrophe

#21

Post by krapp »

jetrink wrote: post #7
unnouinceput wrote: post #4

The article's title is "How we could stumble into AI catastrophe". HN title is a clickbait

I believe HN automatically removes 'how' from the start of titles on submission.

A lot of these edits HN makes to titles seem to do more harm than good. I wonder what evidence they have, if any, to the contrary - that they improve the quality of conversation in any way?

On topic, I wonder if ChatGPT could be trained to generate a non-clickbaity, accurate headline and title for submissions? Can it be trained either to gauge bias or remove it?

Re: We could stumble into AI catastrophe

#22

Post by dwohnitmok »

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans.

This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both digital and physical, to give them as much tangible impact as possible.

With the rapid rise of AI capabilities, where groundbreaking advances are being measured in weeks rather than years, I strongly urge developers to think harder about AI safety. AI safety shouldn't be a dirty word that has quotes surrounding it to delegitimize it as a farfetched concern. Benchmarks of things that we thought AIs couldn't do or were years if not decades out are falling by the day.

Re: We could stumble into AI catastrophe

#23

Post by Oxidation »

DerekBickerton wrote: post #5

AI will always need humans in the loop to intervene. Remember the old IBM motto: 'A computer isn't accountable so a computer should never make a management decision'.

"Should never" doesn't mean "will never", and one company's training slide doesn't have any more practical weight than Asimov's Laws do.

Besides, it's predicted on accountability being something a large company would want. There's already a whole industry around accountability-washing as it is: management consultancy.

Re: We could stumble into AI catastrophe

#24

Post by rnk »

DerekBickerton wrote: post #5

AI will always need humans in the loop to intervene. Remember the old IBM motto: 'A computer isn't accountable so a computer should never make a management decision'.

That's extremely naive. There's no leader of major companies today that would follow that in practice. They'd probably brag about doing the opposite (scaling our management by using AI to make easy decisions).

Re: We could stumble into AI catastrophe

#25

Post by arbuge »

ronsor wrote: post #6
unnouinceput wrote: post #4

The article's title is "How we could stumble into AI catastrophe". HN title is a clickbait

HN strips off the "How" by default.

I could see some things going wrong with that.

For example, suppose I submitted a post titled "How X Works Now.". Eg. "How Affiliate Marketing Works Now.".

Removing the "How" wouldn't really convey the sense intended.

Re: We could stumble into AI catastrophe

#27

Post by [deleted] »

Oxidation wrote: post #23
DerekBickerton wrote: post #5

AI will always need humans in the loop to intervene. Remember the old IBM motto: 'A computer isn't accountable so a computer should never make a management decision'.

"Should never" doesn't mean "will never", and one company's training slide doesn't have any more practical weight than Asimov's Laws do. Besides, it's predicted on accountability being something a large company would want. There's already a whole industry around accountability-washing as it is: management consultancy.

[deleted]

Re: We could stumble into AI catastrophe

#28

Post by ajuc »

dwohnitmok wrote: post #22

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

Corporations are AIs implemented on brains. And they are profit-maximizers. And they routinely do A LOT of damage. They even have legal rights :)

We're just digitizing that.

Re: We could stumble into AI catastrophe

#29

Post by Animats »

This author is mostly writing an update of 1950s science fiction about a robot takeover.

Things to worry about in the near term:

- Really effective targeted advertising. Most computers already have a camera watching you. That's now coming to TV sets. Amazon and Google are always listening. So far, all this info is only used to select ads. Soon, it should be possible to generate customized marketing content for each consumer, and use immediate feedback on how the customer reacts to adjust the sales pitch. Alexa is already part of the way there.

- Machines should think, people should work. That's what working in an Amazon warehouse is like today. The machines tell the humans what to do. They have to; only the computers have an overview of the process. (Yeah, Marshall Brain's "Manna", which everyone here has probably read.)

- Big Brother is watching you. It is possible now to watch most of the people most of the time. Track who's out of view and for how long, to know what's being missed. China leads in this, but the UK is not far behind. The US isn't centralized enough to integrate all the available data yet. We're coming up on Oppression 2.0. Latest advance - Iran is using surveillance cameras and face recognition to catch women not wearing hijabs.

- Machines beating humans at business. In some areas, AI systems may generate better returns than humans. That already happens in parts of finance. After all, it's an optimization problem. The free market may force companies to use AI more in management.

- Reduced need for education. Right now, about half of college graduates do jobs that don't require a college education. For many people, going to college is not cost-effective. That will increase.

This is all next 5 to 10 years stuff.

Re: We could stumble into AI catastrophe

#30

Post by Animats »

dwohnitmok wrote: post #22

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans. This is especially the case because it seems like private individuals and companies are racing to hook up AIs to all sorts of real world systems, both…

An important point is that AI does not need to be sentient or conscious to do a lot of damage. An AI that is simply very good at optimizing a particular task can do a significant amount of damage inadvertently, if it turns out that task in the limit is bad for humans.

We have that now. It's called a corporation. See yesterday's article about Exxon.

Post reply on HN