Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why do we tell ourselves scary stories about AI?

quantamagazine.org

Post reply on HN
Print view View original

21–30 of 133 posts

Re: Why do we tell ourselves scary stories about AI?

#21

Post by 0x4e »

nalekberov wrote: post #11

> “The last four years have demonstrated that AI agents can acquire the will to survive and that AIs have already learned how to lie.” Why Harari feels an obligation to comment about everything is of course beyond me, but describing 'AI' as if it takes independent decisions to lie, make moral judgements, etc. demonstrates either that he has zero clue how 'AI' trains itself or that he chooses to mislead the audience.

Agreed. But, could it be trained ti be deceiving? Especially when we bake-in advertising into it?

Re: Why do we tell ourselves scary stories about AI?

#22

Post by ACCount37 »

everdrive wrote: post #17

One thing that strikes me that I never really see anyone discuss is that we've been afraid of conscious computers for a _long_ time. Back in the 50s and before people were quite afraid that we'd build conscious computers. This was long before there was any sense that could actually accomplish the task. I think that similarly to seeing faces in the clouds, we imagine a consciousness where none exists. (eg: a rain god…

Are they? Not conscious?

If you list out every prominent theory of consciousness, you'd find that about a quarter rules out LLMs, a quarter tentatively rules LLMs in, and what remains is "uncertain about LLMs". And, of course, we don't know which theory of consciousness is correct - or if any of them is.

So, what is it that makes you so sure, oh so very certain, that LLMs just "feel" conscious but aren't?

Re: Why do we tell ourselves scary stories about AI?

#23

Post by yoz-y »

nalekberov wrote: post #11

> “The last four years have demonstrated that AI agents can acquire the will to survive and that AIs have already learned how to lie.” Why Harari feels an obligation to comment about everything is of course beyond me, but describing 'AI' as if it takes independent decisions to lie, make moral judgements, etc. demonstrates either that he has zero clue how 'AI' trains itself or that he chooses to mislead the audience.

Isn't the problem precisely that it does not take moral judgements?

My opinion on all of this is constantly shifting, but right now my main issue is that-like self driving-it seems 90-95% correct and 5-10% catastrophically wrong.

Due to the sheer speed and volume of output it produces I have grown complacent and exhausted, so when I give it simple tasks I assume it is correct and then is the time when "it deletes" all of your files.

Re: Why do we tell ourselves scary stories about AI?

#26

Post by SpicyLemonZest »

The actual contents of this article are making reasonable arguments I largely agree with. It would be very surprising for LLM-based AI systems to act as monomanaical goal optimizers, since they're trained on human text and humans are extremely bad at goal-oriented behavior. (My goals for today include a number of work and self-maintenance tasks, and the time I'm spending here writing out a HN comment does not all help achieve them - I suspect most people reading this comment are in the same boat.)

It's very frustrating that the magazine wrote such a dumb headline which guarantees people won't talk about the issues the article raised. Obviously non-goal-oriented systems can still have important negative effects.

Re: Why do we tell ourselves scary stories about AI?

#27

Post by MisterTea »

afavour wrote: post #3

It does feel like a bizarre moment, where the AI companies are deliberately trying to scare us about their own product in a bid to, I think, show the inevitability of it? Or to sell themselves as the one responsible power to constrain it? It's very odd. "It's going to take all your jobs" is not a great selling point to the everyday public.

> It does feel like a bizarre moment, where the AI companies are deliberately trying to scare us about their own product

That is direct CEO to CEO marketing. They're working really hard to convince high up decision makers that these tools will lower their head count and reduce costs.

Re: Why do we tell ourselves scary stories about AI?

#30

Post by 0x4e »

bpodgursky wrote: post #18
afavour wrote: post #3

It does feel like a bizarre moment, where the AI companies are deliberately trying to scare us about their own product in a bid to, I think, show the inevitability of it? Or to sell themselves as the one responsible power to constrain it? It's very odd. "It's going to take all your jobs" is not a great selling point to the everyday public.

They are being honest and you don't want to deal with the implications, so you stretch for conspiracy theories. The ones at the top are the true believers. Engage with them at that level.

That's a good view point. Perhaps they're not being alarmists or trying to scare people, but being honest about the capabilities.

Perhaps it can be better articulated and framed in a way that's well received. But, maybe that would be over-promising or not being honest about the future.

Post reply on HN