Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why do we tell ourselves scary stories about AI?

quantamagazine.org

Post reply on HN
Print view View original

121–130 of 133 posts

Re: Why do we tell ourselves scary stories about AI?

#121

Post by mastermage »

bigbadfeline wrote: post #117
mastermage wrote: post #116

Earlier quoted context omitted.

I think the Nuclear Bomb is still scarier. But AI is not scary for its destructive potential but for its potential to disrupt our society fundamentally, and not just in a good way.

> I think the Nuclear Bomb is still scarier. But AI is not scary for its destructive potential AI excels in both making weapons of all kinds and effectively targeting them, as the resent war has shown - AI is more dangerous, and can be more destructive, than all weapons taken together.

I meant more in a physical destructive sense.

Re: Why do we tell ourselves scary stories about AI?

#122

Post by dryarzeg »

Nasrudith wrote: post #115
dryarzeg wrote: post #67

Earlier quoted context omitted.

> technocracy What you mentioned is not a technocracy. Technocracy is when all decisions are made by real specialists in the field, based on scientific methods (simply speaking). What you mentioned is a plutocracy, a form of oligarchy in which decisions are made by people of great wealth. I couldn’t just ignore this because, in my view, technocracy (as I’ve described it) still has some merit - for instance, appointin…

Technocracy is a myth thanks to Goodhart's Law. Whomever sets the measures just becomes the new dictator as they will continue insist on whatever doctrinal point until they retain power. Technocracy would just end being the same old 'push delusions to remain in power' government as fascism and theocracy.

> Technocracy is a myth

"Complete" technocracy - yes, although in real life I can’t recall any states that have actually implemented it. But I didn’t say that technocracy is the solution to all our problems. I merely said the following, and I quote: "still has some merit". In other words, there is at least something good about it; there's some elements that could be adopted. Or, in your view, would it be better if the state were run by idiots who don’t even understand what, in theory, is their job?

Re: Why do we tell ourselves scary stories about AI?

#123

Post by sharlos201068 »

djhn wrote: post #120
ACCount37 wrote: post #79

Earlier quoted context omitted.

Is it me making the mistake, or is it you making that very mistake in the other direction? Back in the "mainframe era", we had entire lists of tasks that even the most untrained humans would find trivial, but computers were impossibly bad at. Like following informal instructions, or telling a picture of a dog from that of a cat. We're in the "AI era" now, and what remains of those lists? What are the areas of human a…

The success rate of computers doing those tasks has gone from 0% to 70%-90%, but reaching 100% might take a very long time.

How many people are needed to make up the difference to 100% though?

Re: Why do we tell ourselves scary stories about AI?

#124

Post by sharlos201068 »

ThrowawayR2 wrote: post #73
ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

There was a lot of FUD in the mainframe era about computers being called "electronic brains" and fears of them taking people's jobs because the ignorant public mistook their lighting fast arithmetic skills for intelligence. Many did lose their jobs as digital record keeping, computerized accounting/ERP, robotics on assembly lines, became cost effective, but at no time did the "electronic brain" become intelligent. Th…

They don't need to be sapient to be dangerous though

Re: Why do we tell ourselves scary stories about AI?

#125

Post by tim333 »

>After talking to experts, I was convinced there’s no reason to fear AIs developing a will to live, and then tricking or destroying us to avoid shutdown and take over the world. Unless, of course, we tell them to.

Once we have super intelligent AIs I give it a day or two till someone tries that, prompting something like take over the world for me.

Re: Why do we tell ourselves scary stories about AI?

#126

Post by deterministic »

ACCount37 wrote: post #19

It's simple. It's because AI is the scariest technology ever made. Human intelligence has proven itself capable of doing a lot of scary things. And AI research is keen on building ever more capable and more scalable intelligence. By now, the list of advantages humans still have over machines is both finite and rapidly diminishing. If that doesn't make you uneasy, you're in denial.

Hmmm I would personally pick nuclear weapons as the #1 scary tech.

And a close (non-tech) second is the ruthlessness of sociopaths seeking power.

Re: Why do we tell ourselves scary stories about AI?

#127

Post by deterministic »

ACCount37 wrote: post #72
afavour wrote: post #38

Earlier quoted context omitted.

I don't know, I think nuclear weapons are scarier. And also probably a useful parallel: they're so dangerous that we coined the term "mutually assured destruction" and everyone recognized that it was so dangerous to use them that they've only ever been used once. I see the flood of PR from AI firms as an attempt to make sure we don't build the appropriate safeguards this time around, because there's too much money to…

I remind you of why nuclear weapons exist. They exist because human minds conceived them, and human hands made them. One of the major dangers of advanced AI is being able to implement something not unlike Manhattan project with synthetic intelligence, in a single datacenter.

You can't create and enrich nuclear materials inside a datacenter.

Re: Why do we tell ourselves scary stories about AI?

#129

Post by qgin »

We’ve seen an incredibly powerful technology follow multiple exponential curves in its capability, but we’re supposed to ask why we’re telling ourselves “stories” if we think about what will happen if that technology continues to follow the curves it has been following without sign of hitting any walls?

Is AGI certain? No. But there’s currently no specific reason to believe it isn’t coming in the next few years.

Re: Why do we tell ourselves scary stories about AI?

#130

Post by dclowd9901 »

SpicyLemonZest wrote: post #70
dclowd9901 wrote: post #69

Earlier quoted context omitted.

Because they don't _understand_ things. If I teach an LLM that 3+5 is 8, it doesn't "get" that 4+5 is 9 (leave aside the details here, as I'm explaining for effect). It needs to be taught that as well, and so on. We understand exactly everything that goes into how LLMs generate answers. The line of consciousness, as we understand it, is understanding. And as far as what actually constitutes consciousness, we're not e…

> Because they don't _understand_ things. If I teach an LLM that 3+5 is 8, it doesn't "get" that 4+5 is 9 (leave aside the details here, as I'm explaining for effect). It needs to be taught that as well, and so on. We understand exactly everything that goes into how LLMs generate answers. What you're saying just isn't true, even directionally. Deployed LLMs routinely generalize outside of their training set to apply…

How is it not true? Theres a world of difference between predicting the next word of a sentence in a summary and understanding the tenets of mathematics. You're mistaking general application of mathematical knowledge with memorization of mathematical outcomes.
Post reply on HN