Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why do we tell ourselves scary stories about AI?

quantamagazine.org

Post reply on HN
Print view View original

51–60 of 133 posts

Re: Why do we tell ourselves scary stories about AI?

#51

Post by saHqtr »

jacquesm wrote: post #40
ACCount37 wrote: post #22

Earlier quoted context omitted.

Are they? Not conscious? If you list out every prominent theory of consciousness, you'd find that about a quarter rules out LLMs, a quarter tentatively rules LLMs in, and what remains is "uncertain about LLMs". And, of course, we don't know which theory of consciousness is correct - or if any of them is. So, what is it that makes you so sure, oh so very certain, that LLMs just "feel" conscious but aren't?

The fact that it's a box with a plug and a state that can be fully known. A conscious entity has a state that can not be fully known. Far smarter people than me have made this argument and in a much more eloquent way. Turing aimed too low.

And the chatbots don't even pass the Turing test.

I've never had a normal conversation. It's always prompt => lengthy, cocksure and somewhat autistic response. They are very easily distinguishable.

Re: Why do we tell ourselves scary stories about AI?

#53

Post by xg15 »

everdrive wrote: post #17

One thing that strikes me that I never really see anyone discuss is that we've been afraid of conscious computers for a _long_ time. Back in the 50s and before people were quite afraid that we'd build conscious computers. This was long before there was any sense that could actually accomplish the task. I think that similarly to seeing faces in the clouds, we imagine a consciousness where none exists. (eg: a rain god…

The idea of "artificial beings" in some way or another seems to have been with humanity for a long time already: https://en.wikipedia.org/wiki/Golem

Re: Why do we tell ourselves scary stories about AI?

#54

Post by MyHonestOpinon »

everdrive wrote: post #17

One thing that strikes me that I never really see anyone discuss is that we've been afraid of conscious computers for a _long_ time. Back in the 50s and before people were quite afraid that we'd build conscious computers. This was long before there was any sense that could actually accomplish the task. I think that similarly to seeing faces in the clouds, we imagine a consciousness where none exists. (eg: a rain god…

It is so interesting how in the 50s we "felt" that AI was possible even if we didn't even have the slightest idea on how that would work. Later on, when we started to understand computers it looked like a very remote possibility in the far future, something our great grand kids may need to worry about. And suddenly it is here and the dangers seem a lot more real.

Re: Why do we tell ourselves scary stories about AI?

#55

Post by RivieraKid »

I 100% agree with this take, I find AI completely non-scary, especially in the sense that it's some kind of a conscious entity that will want to take over. I find these people almost delusional. It's a powerful tool, so it can be dangerous if used by people with bad intentions, so there's some real danger here, but my intuition is that it will be fine. The ratio between the power of people with good vs bad intentions shouldn't change too dramatically.

The only scary part is that it could be bad for my future as a software developer. That said, I think it will be net benefit for the average worker - the average person will work less and earn more.

Re: Why do we tell ourselves scary stories about AI?

#56

Post by webdoodle »

afavour wrote: post #38
ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

I don't know, I think nuclear weapons are scarier. And also probably a useful parallel: they're so dangerous that we coined the term "mutually assured destruction" and everyone recognized that it was so dangerous to use them that they've only ever been used once. I see the flood of PR from AI firms as an attempt to make sure we don't build the appropriate safeguards this time around, because there's too much money to…

Nuclear weapons have rarely been used kinetically. Their real force multiplier is the fear.

A.I. is being used by so many people for so many diabolical things, hidden, unknown things that we may never fully understand it's purpose. But that doesn't mean it's purpose won't destroy us in the end.

The expression "Drinking the Koolaid" is used to explain the Jonestown mass suicide. It is an information hazard, aka, a cult that created the end result: 900 people drinking poisoned flavoraid. That's just one example of a human caused information hazard. What happens when someone with similar thinking applies that to A.I.? Will we even be able to sleuth out who did it?

Re: Why do we tell ourselves scary stories about AI?

#57

Post by cbogie »

FatherOfCurses wrote: post #36

We tell ourselves scary stories about AI because humanity is rife with stories where a new idea or technology has had unintended negative consequences. AI bros just care about selling their product to another company and cashing out, they have absolutely no regard for their legacy.

we tell ourselves scary stories because that’s what we do with language. and now our language isn’t just ours anymore.

Re: Why do we tell ourselves scary stories about AI?

#58

Post by andrewmutz »

ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

Modern discourse happens on social media where fear and outrage drive engagement, which drives virality. We have become convinced in a short amount of time that AI is going to take all the jobs and eventually kill us all because that's what people click on.

Any voices or studies that present the case for "useful technology that will improve productivity and wages while not murdering us" don't get clicked on or read.

Re: Why do we tell ourselves scary stories about AI?

#59

Post by dclowd9901 »

My favorite part of this article was this bit, and naturally so, since I love the author:

> Where did we come up with this caricature of AI’s obsessive rationality? “There’s an article I love by [the sci-fi author] Ted Chiang,” Mitchell said, “where he asks: What entity adheres monomaniacally to one single goal that they will pursue at all costs even if doing so uses up all the resources of the world? A big corporation. Their single goal is to increase value for shareholders, and in pursuing that, they can destroy the world. That’s what people are modeling their AI fantasies on.” As Chiang put it in the article in The New Yorker(opens a new tab), “Capitalism is the machine that will do whatever it takes to prevent us from turning it off.”

I didn't realize it til I read it here, but yes, my fear isn't really about the machine, it's about the machine that drives the machine. We already have a class of amoral beings that treat the world as an expendable thing and are willing to burn it down for profit. We should focus on getting rid of that problem first.

Re: Why do we tell ourselves scary stories about AI?

#60

Post by psychoslave »

ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

Machine still need a planetary complex production pipelines with human operators everywhere to achieve reproduction at scale. Even taking paperclip plant optimizer overlord as serious scenario, it’s still several order of magnitude less likely than humans letting the most nefarious individuals create international conflicts and engage in genocides, not even talking about destroying vast pans of the biosphere supporting humanity possibility of existence.

That is, also alien invasion and giant meteor are plausible scenario, but at some point one has to prioritize threats likeliness, and generally speaking it makes more sense to put more weight on "ongoing advanced operation" than on "not excluded in currently known scientifically realistic what-if".

Post reply on HN