Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why do we tell ourselves scary stories about AI?

quantamagazine.org

Post reply on HN
Print view View original

81–90 of 133 posts

Re: Why do we tell ourselves scary stories about AI?

#81

Post by sifar »

ACCount37 wrote: post #76
fontain wrote: post #49

Earlier quoted context omitted.

The world we live in is a construct, not a natural outcome. Even if we take your premise at face value, that our success as a species is only because of advantages over others, what's to say that "intelligence" is that advantage? What's to say that we don't use our advantages to reconstruct a world that works in a way that doesn't advantage intelligence over all else? And on intelligence specifically: even amongst th…

IQ is among the best predictors of life success, for humans. Being up by an extra SD in the g dimension covers a multitude of sins. I'm not sure what level of delusion one has to run to look at human civilization and say "no, intelligence wasn't important for this". It's pretty obvious that human world is a product of intelligence applied at scale - and machines can beat humans at intelligence and scale both.

>> I'm not sure what level of delusion one has to run to look at human civilization and say "no, intelligence wasn't important for this".

One has to only look at the current tech and political leaders.

Re: Why do we tell ourselves scary stories about AI?

#82

Post by lopsotronic »

afavour wrote: post #38
ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

I don't know, I think nuclear weapons are scarier. And also probably a useful parallel: they're so dangerous that we coined the term "mutually assured destruction" and everyone recognized that it was so dangerous to use them that they've only ever been used once. I see the flood of PR from AI firms as an attempt to make sure we don't build the appropriate safeguards this time around, because there's too much money to…

By any quantifiable measure, yes, and not by small numbers either.

Until someone can demonstrate a quantitative measure of intelligence - with the same stability of measurement as "meters" or "joules" - any discussion of "Super-AI" as "the most dangerous X" is at best qualitative/speculative risk narratology, at worst discursive distractions. The architecture of the "social web" amplifies discursion to a harmful degree in an open population of agents, something I think we could probably prove mathematically. I am more suspicious of this social principle than I am scared of Weakly Godlike Intelligence at this moment in history; I am more scared of nuclear weapons than literally anything else.

People think we are out of the woods with nuclear weapons, but I don't think we've even seen the forest yet. We are Homo Erectus, puffing on a flame left by a lightning strike, carrying this magic fire back to our cave.

Re: Why do we tell ourselves scary stories about AI?

#83

Post by everdrive »

ACCount37 wrote: post #22
everdrive wrote: post #17

One thing that strikes me that I never really see anyone discuss is that we've been afraid of conscious computers for a _long_ time. Back in the 50s and before people were quite afraid that we'd build conscious computers. This was long before there was any sense that could actually accomplish the task. I think that similarly to seeing faces in the clouds, we imagine a consciousness where none exists. (eg: a rain god…

Are they? Not conscious? If you list out every prominent theory of consciousness, you'd find that about a quarter rules out LLMs, a quarter tentatively rules LLMs in, and what remains is "uncertain about LLMs". And, of course, we don't know which theory of consciousness is correct - or if any of them is. So, what is it that makes you so sure, oh so very certain, that LLMs just "feel" conscious but aren't?

This isn't meant to be an answer that would satisfy everyone, but in my opinion consciousness is a specific biological / evolutionary adaptation that has to do with managing status, relationships, and caring for young. It's about having an identity and an ego and building mental models of the egos / identities / etc of others.

I don't think there's any reason we couldn't in principle attach this sort of concept to an LLM, but it's not something we've actually done. (and no, prompting an LLM to act as if it has an identity does not count)

Re: Why do we tell ourselves scary stories about AI?

#84

Post by dryarzeg »

SpicyLemonZest wrote: post #74
dryarzeg wrote: post #67

Earlier quoted context omitted.

> technocracy What you mentioned is not a technocracy. Technocracy is when all decisions are made by real specialists in the field, based on scientific methods (simply speaking). What you mentioned is a plutocracy, a form of oligarchy in which decisions are made by people of great wealth. I couldn’t just ignore this because, in my view, technocracy (as I’ve described it) still has some merit - for instance, appointin…

I think most AI execs I'm familiar with would , if they were the god-monarch of humanity, recruit real specialists applying scientific methods to make most decisions. They seem like the kind of people who would understand that the Ministry of Economy is doing valuable things which shouldn't be compromised for personal expediency. Does that really make it any better?

> if they were the god-monarch of humanity

In that case, we're not talking about an oligarchy or a technocracy either. What you have described is an autocracy - a rule by one. When there's some kind of "god-monarch", the people heading the Ministry of Economy will be controlled by this "god-monarch" and it's unclear if this can be called technocracy or not (at least it is unclear to me; maybe I'm stupid, who knows).

> Does that really make it any better?

Honestly? If you're asking about "would it be any better than now" - I'm not really sure, because I'm not in power to access the actions of the people who hold the positions equivalent to the head of Ministry of Economy - the economy is not my field, I'm not a specialist here. I would only point to the example I'm familiar with (and you're probably not; I'm sorry, I just couldn't think about something like this that I can verify) - in Ukraine, there's a "Ministry of Digital Transformation". This ministry was headed by a Mykhailo Fedorov, whose primary and as far as I know, he studied at the "Faculty of Sociology and Management". Well, that’s not the main point, as he’s studied elsewhere too. The problem lies elsewhere. His decisions have been criticized on more than one occasion by genuine experts - for example, the project known as "Diya", or "the state in a smartphone"; in short, it’s something like access to documents and various government services all in one app. It’s a long story… In short, as a result, there were (presumably) data leaks, and the service crashed more than once or twice due to its flawed security, and all sorts of problems were found with it - you name it, it had it. It's such a shame, to be honest... You can't just go and play with things like that. And now that person is serving as a... head of Ministry of Defence. Hell. To add insult to the injury, guess who is now taking his place at a Ministry of Digital Transformation? Oleksandr Bornyakov, who, as far as I know, holds a degree in marketing. Marketing. ...Nice. Well maybe I don't know something, who knows, who knows... but the decisions, or, rather, their consequences seem to be... let's settle with "terrible".

I'm pretty sure you can recall some similar examples yourself. My point is that although the scenario you described is not as good - because, I guess, no one really wants "god-monarch" controlling (although not making directly) all of the decisions. But if our hypothetical Ministry of the Economy were run by genuine experts who, moreover, work for the good of society - or at least the state as a whole - rather than just lining their own pockets, well, that sounds better than an idiocracy. That was my point.

Re: Why do we tell ourselves scary stories about AI?

#85

Post by everdrive »

altairprime wrote: post #65
everdrive wrote: post #17

One thing that strikes me that I never really see anyone discuss is that we've been afraid of conscious computers for a _long_ time. Back in the 50s and before people were quite afraid that we'd build conscious computers. This was long before there was any sense that could actually accomplish the task. I think that similarly to seeing faces in the clouds, we imagine a consciousness where none exists. (eg: a rain god…

That same fear is directed towards human sociopathy, as much of entire thriller genre indicates. It turns out that most people carry a specific duality: first, they’re deathly afraid of being unable to socially pressure other beings into being good citizens — whether due to asocial, or alien, or monstrous, or corrupted; and second, they’re excited to celebrate when people reach their breaking point and stop being goo…

>That same fear is directed towards human sociopathy, as much of entire thriller genre indicates.

This is a great insight, and I think in general people have a pretty broken view of what sociopathy is.

Re: Why do we tell ourselves scary stories about AI?

#86

Post by grantcas »

ACCount37 wrote: post #22
everdrive wrote: post #17

One thing that strikes me that I never really see anyone discuss is that we've been afraid of conscious computers for a _long_ time. Back in the 50s and before people were quite afraid that we'd build conscious computers. This was long before there was any sense that could actually accomplish the task. I think that similarly to seeing faces in the clouds, we imagine a consciousness where none exists. (eg: a rain god…

Are they? Not conscious? If you list out every prominent theory of consciousness, you'd find that about a quarter rules out LLMs, a quarter tentatively rules LLMs in, and what remains is "uncertain about LLMs". And, of course, we don't know which theory of consciousness is correct - or if any of them is. So, what is it that makes you so sure, oh so very certain, that LLMs just "feel" conscious but aren't?

[dead]

Re: Why do we tell ourselves scary stories about AI?

#87

Post by AlecSchueler »

andrewmutz wrote: post #58
ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

Modern discourse happens on social media where fear and outrage drive engagement, which drives virality. We have become convinced in a short amount of time that AI is going to take all the jobs and eventually kill us all because that's what people click on. Any voices or studies that present the case for "useful technology that will improve productivity and wages while not murdering us" don't get clicked on or read.

> Any voices or studies that present the case for "useful technology that will improve productivity and wages while not murdering us" don't get clicked on or read.

It's not an either/or thing though. Compare to something like combustion. Sure it definitely improved productivity but also lead to countless violent deaths.

Re: Why do we tell ourselves scary stories about AI?

#88

Post by altairprime »

everdrive wrote: post #85
altairprime wrote: post #65

Earlier quoted context omitted.

That same fear is directed towards human sociopathy, as much of entire thriller genre indicates. It turns out that most people carry a specific duality: first, they’re deathly afraid of being unable to socially pressure other beings into being good citizens — whether due to asocial, or alien, or monstrous, or corrupted; and second, they’re excited to celebrate when people reach their breaking point and stop being goo…

>That same fear is directed towards human sociopathy, as much of entire thriller genre indicates. This is a great insight, and I think in general people have a pretty broken view of what sociopathy is.

I think that’s because in general people have a pretty broken view of the specific cognitive dissonance that 1) sociopaths can choose to disregard ‘polite’ societal boundaries, and 2) sociopaths are invaluable for disregarding ‘polite’ societal boundaries. Consider four movies with sociopathic leads: “American Psycho” and “Ferris Bueller”, versus “John Wick” and “Jennifer’s Body”; if you try to define their sociopathy in terms of U.S. Christianity’s Good Versus Evil, you’ll end up smashing into dissonances galore. But since having a small fraction of a society be sociopathic is highly beneficial, as it serves as random enforcement of societal laws against grifts from within, you end up with the fear/fame duality we see in media; sociopaths can be absurdly charismatic, and even more prosocial than the best of those who are not asocial.

This suggests a very interesting point, one that makes people deeply uncomfortable:

People are afraid of AI going Skynet on us because, when the roles are swapped, we cheer for John Wick’s prosocial enforcement — and deep down, people believe we deserve more prosocial enforcement for how our societies treat us.

Re: Why do we tell ourselves scary stories about AI?

#89

Post by jaredcwhite »

ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

AI, the way you are describing it, has not been invented yet. It is a fiction.

What is called "AI" today is an extremely vague marketing term being applied to various software technologies which are only dangerous because humans are dangerous. Nuclear & chemical weapons are also "very scary" but only because the humans who might decide to use them in fits of insanity are scary.

I'm not in the slightest bit uneasy about "AI" itself right now, because as I said, the AI of Sci-Fi has not yet been invented…and seems unlikely to in any of our lifetimes. (Not throwing shade on clever researchers. We also don't have working FTL travel, though plenty of scientists speculate on how such an engine might be built.)

Re: Why do we tell ourselves scary stories about AI?

#90

Post by jaredcwhite »

bauerd wrote: post #47
sublinear wrote: post #37

Earlier quoted context omitted.

This is untrue. What is being diminished is the value of humans doing repetitive or uncreative tasks. Many have built their careers from that kind of work in the past and yes they are threatened, but that kind of work is inherently not collaborative and more vocational.

The vast majority of people on this planet work repetitive, uncreative jobs.

Oh? And what extensive knowledge and experience makes YOU qualified to determine what "the vast majority of people on this planet" are doing for work and if those tasks are creative or uncreative?
Post reply on HN