Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why do we tell ourselves scary stories about AI?

quantamagazine.org

Post reply on HN
Print view View original

71–80 of 133 posts

Re: Why do we tell ourselves scary stories about AI?

#71

Post by otabdeveloper4 »

ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

> AI is the scariest technology ever made

Well, it's a good thing that all we managed so far is a large language model instead.

Re: Why do we tell ourselves scary stories about AI?

#72

Post by ACCount37 »

afavour wrote: post #38
ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

I don't know, I think nuclear weapons are scarier. And also probably a useful parallel: they're so dangerous that we coined the term "mutually assured destruction" and everyone recognized that it was so dangerous to use them that they've only ever been used once. I see the flood of PR from AI firms as an attempt to make sure we don't build the appropriate safeguards this time around, because there's too much money to…

I remind you of why nuclear weapons exist.

They exist because human minds conceived them, and human hands made them.

One of the major dangers of advanced AI is being able to implement something not unlike Manhattan project with synthetic intelligence, in a single datacenter.

Re: Why do we tell ourselves scary stories about AI?

#73

Post by ThrowawayR2 »

ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

There was a lot of FUD in the mainframe era about computers being called "electronic brains" and fears of them taking people's jobs because the ignorant public mistook their lighting fast arithmetic skills for intelligence. Many did lose their jobs as digital record keeping, computerized accounting/ERP, robotics on assembly lines, became cost effective, but at no time did the "electronic brain" become intelligent.

There's a lot of FUD today about LLM's being sapient because the ignorant public mistakes their complex token prediction skills for intelligence. But it's just embarrassing to see people making that mistake on a forum ostensibly filled with hackers.

Re: Why do we tell ourselves scary stories about AI?

#74

Post by SpicyLemonZest »

dryarzeg wrote: post #67
0x4e wrote: post #14

Because we don't like uncertainty, and the AI future is uncertain. There are multiple high probability scenarios. Because we're seeing how its capabilities increase overtime. I find the rate at which I prefer to go to an AI than an UpWorker is scary. Because we——the people——are not in control of it. We're at the whims of whatever it and the tech bros want (technocracy).

> technocracy What you mentioned is not a technocracy. Technocracy is when all decisions are made by real specialists in the field, based on scientific methods (simply speaking). What you mentioned is a plutocracy, a form of oligarchy in which decisions are made by people of great wealth. I couldn’t just ignore this because, in my view, technocracy (as I’ve described it) still has some merit - for instance, appointin…

I think most AI execs I'm familiar with would, if they were the god-monarch of humanity, recruit real specialists applying scientific methods to make most decisions. They seem like the kind of people who would understand that the Ministry of Economy is doing valuable things which shouldn't be compromised for personal expediency. Does that really make it any better?

Re: Why do we tell ourselves scary stories about AI?

#75

Post by ACCount37 »

psychoslave wrote: post #60
ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

Machine still need a planetary complex production pipelines with human operators everywhere to achieve reproduction at scale. Even taking paperclip plant optimizer overlord as serious scenario, it’s still several order of magnitude less likely than humans letting the most nefarious individuals create international conflicts and engage in genocides, not even talking about destroying vast pans of the biosphere supporti…

Humans are dangerous and hilariously exploitable.

If politicians can get away with what they do? Imagine if those politicians were actually smart and diligent to a superhuman degree.

That's the kind of threat a rogue AI can pose.

Humans can easily act against their own self-interest. If other humans can and evidently do exploit that, what would stop something better than human from doing the same?

Re: Why do we tell ourselves scary stories about AI?

#76

Post by ACCount37 »

fontain wrote: post #49
ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

The world we live in is a construct, not a natural outcome. Even if we take your premise at face value, that our success as a species is only because of advantages over others, what's to say that "intelligence" is that advantage? What's to say that we don't use our advantages to reconstruct a world that works in a way that doesn't advantage intelligence over all else? And on intelligence specifically: even amongst th…

IQ is among the best predictors of life success, for humans. Being up by an extra SD in the g dimension covers a multitude of sins.

I'm not sure what level of delusion one has to run to look at human civilization and say "no, intelligence wasn't important for this". It's pretty obvious that human world is a product of intelligence applied at scale - and machines can beat humans at intelligence and scale both.

Re: Why do we tell ourselves scary stories about AI?

#77

Post by ACCount37 »

saHqtr wrote: post #45
ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

Most humans can do more than plagiarizing text. But let's hype up the clankers before the IPOs.

"It's all just PR" is a lame excuse not to think about the implications. Of things like: AI capabilities only ever going up over the course of the past 4 years.

Re: Why do we tell ourselves scary stories about AI?

#78

Post by ACCount37 »

dclowd9901 wrote: post #69
ACCount37 wrote: post #22

Earlier quoted context omitted.

Are they? Not conscious? If you list out every prominent theory of consciousness, you'd find that about a quarter rules out LLMs, a quarter tentatively rules LLMs in, and what remains is "uncertain about LLMs". And, of course, we don't know which theory of consciousness is correct - or if any of them is. So, what is it that makes you so sure, oh so very certain, that LLMs just "feel" conscious but aren't?

Because they don't _understand_ things. If I teach an LLM that 3+5 is 8, it doesn't "get" that 4+5 is 9 (leave aside the details here, as I'm explaining for effect). It needs to be taught that as well, and so on. We understand exactly everything that goes into how LLMs generate answers. The line of consciousness, as we understand it, is understanding. And as far as what actually constitutes consciousness, we're not e…

Leave aside "the details" like you being obviously, provably wrong?

We've known for a long while that even basic toy-scale AIs can "grok" and attain perfect generalization of addition that extends to unseen samples.

Humans generalize faster than most AIs, but AIs generalize too.

Re: Why do we tell ourselves scary stories about AI?

#79

Post by ACCount37 »

ThrowawayR2 wrote: post #73
ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

There was a lot of FUD in the mainframe era about computers being called "electronic brains" and fears of them taking people's jobs because the ignorant public mistook their lighting fast arithmetic skills for intelligence. Many did lose their jobs as digital record keeping, computerized accounting/ERP, robotics on assembly lines, became cost effective, but at no time did the "electronic brain" become intelligent. Th…

Is it me making the mistake, or is it you making that very mistake in the other direction?

Back in the "mainframe era", we had entire lists of tasks that even the most untrained humans would find trivial, but computers were impossibly bad at. Like following informal instructions, or telling a picture of a dog from that of a cat.

We're in the "AI era" now, and what remains of those lists? What are the areas of human advantage, the standing bastions of human specialness? Because with modern AI, the list has grown quite thin. Growing thinner as we speak.

Re: Why do we tell ourselves scary stories about AI?

#80

Post by afavour »

SpicyLemonZest wrote: post #66
afavour wrote: post #38

Earlier quoted context omitted.

I don't know, I think nuclear weapons are scarier. And also probably a useful parallel: they're so dangerous that we coined the term "mutually assured destruction" and everyone recognized that it was so dangerous to use them that they've only ever been used once. I see the flood of PR from AI firms as an attempt to make sure we don't build the appropriate safeguards this time around, because there's too much money to…

Everyone recognized that it was so dangerous to use them after the first two mass casualty events. At the time and even into the 50s it was not universally obvious, and the arguments in favor of nuclear weapons use were quite similar to arguments I often see with regards to AI: bombing cities into rubble is not a new concept, traditional explosives well within the supply capacity of large militaries are capable of it…

> Everyone recognized that it was so dangerous to use them after the first two mass casualty events

I really don’t think that’s true. Those who actually knew about the nuclear weapons knew very well how dangerous they were. Truman was deeply conflicted about using them.

Post reply on HN