Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Only You Can Stop an AI Apocalypse

betterwithout.ai

Post reply on HN
Print view View original

81–90 of 130 posts

Re: Only You Can Stop an AI Apocalypse

#81

Post by mistermann »

madrox wrote: post #16

Anti-AI seems poised to be the anti-vax of the next few years, where laymen pick up lines like "we have no idea how AI does the stuff it does" and take that wholly out of context. As a society, we have to decide how we feel about AI in legal and cultural contexts as we expand our capabilities, but there's time to do that. Github Copilot will not end up launching nuclear missiles in the meantime. Stuff like this just…

> As a society, we have to decide how we feel about AI in legal and cultural contexts as we expand our capabilities, but there's time to do that.

How long will it take to learn how to make collective decisions though? There's "democracy" of course, but that has almost nothing to do with making group decisions.

Re: Only You Can Stop an AI Apocalypse

#82

Post by teucris »

The more I read, the more I feel like this is actually a well written piece on why capitalism is causing an apocalypse. Every time the author references AI - that it’s out of control, that it’s influencing major portions of society, that it’s strengthening itself - I think what he actually means is that the corporations are those things, and AI is a tool in their arsenal. He tries to make the argument that it’s the other way around (for a fictional web search company):

> The institution has its own agency: its own purposes, plans, reasons, and logic, which are more powerful than the humans it employs. Those are subordinate in turn to the AI the company depends on for its survival.

The first sentence is absolutely correct. The second? In no way can I believe that the corporation doesn’t have a good understanding of exactly how it can manipulate the AI systems it leverages to achieve its goals.

Re: Only You Can Stop an AI Apocalypse

#83

Post by ShredKazoo »

nhod wrote: post #73

Actually, no, "only you" can't stop it. No one can. There was a book in the 80's called The Hundredth Monkey that posited that if only enough people opposed nuclear technology, there would be a magical critical mass of people after which it would become apparent to everyone in the whole world that nuclear technology was, you know, really bad and stuff, and it would all just go away. We know how that went. Like nuclea…

>We know how that went.

A large majority of the world's countries have no nuclear weapons.

Big swaths of the globe have been designated as "Nuclear-weapon-free zones": https://en.wikipedia.org/wiki/Nuclear-weapon-free_zone

And we haven't had a nuclear apocalypse -- in part due to lots of work on things like weapons treaties, monitoring, control systems, etc.

Throwing up our hands and giving up seems very premature. The challenges are big, but I'd say the importance is high enough that it's worth actually reading this book to see if the author has creative solutions for the things you mention.

Re: Only You Can Stop an AI Apocalypse

#84

Post by eastbound »

Zetobal wrote: post #35
hathawsh wrote: post #30

Today I got a peek into the future. I asked ChatGPT to be the brain for a simple thermostat. I typed the following: "Imagine you are a thermostat for a house HVAC system. The commands you can use to control the HVAC are "HEAT ON", "HEAT OFF", "AC ON", and "AC OFF". I will tell you the temperature and my preferred temperature and you will command the HVAC system. Ready?" ChatGPT happily obliged. I used plain text to t…

Why is everyone bringing examples that are totally explained by how the tokenizer works. There is no intelligence it's Markov chains on crack.

My biggest null hypothesis is that probably half of humans contain “Markov chain not intelligence”. We don’t notice it among engineers, but most humans don’t reason with causal chains but by doing parallels, and they simulate reasonings in a somewhat convincing way. It becomes flagrant in front of a matt exam, a lot of people study a lot but never comprehend maths.

I also often tear my hair when I invite a developer for a programming interview, and they can’t write an if/then/else or string.contains(), despite perfectly simulating answers of advanced developers (which accurately mimic field experts).

Re: Only You Can Stop an AI Apocalypse

#85

Post by ShredKazoo »

teucris wrote: post #82

The more I read, the more I feel like this is actually a well written piece on why capitalism is causing an apocalypse. Every time the author references AI - that it’s out of control, that it’s influencing major portions of society, that it’s strengthening itself - I think what he actually means is that the corporations are those things, and AI is a tool in their arsenal. He tries to make the argument that it’s the o…

>In no way can I believe that the corporation doesn’t have a good understanding of exactly how it can manipulate the AI systems it leverages to achieve its goals.

See this talk which compares modern AI work to alchemy (i.e. it works, unreliably, and we don't always know why): https://www.youtube.com/watch?v=Qi1Yry33TQE

Re: Only You Can Stop an AI Apocalypse

#86

Post by teucris »

chrischen wrote: post #32
jhbadger wrote: post #11

"Most AI researchers think good outcomes are more likely. This seems just blind faith, though" Or, you know, they might actually know something about the subject? On a similar note, most physicists don't think particle colliders will cause mini black holes which will swallow the Earth, but the fear gets repeated by non-physicists.

You are fallaciously appealing to (assumed) authority and missing entirely the fact that the author is a PhD in AI. This critically underscores why blind faith in authority is bad, and why people should do some basic research such as reading the article. By reading the article you would have understood the arguments the author is making on their merits, rather than on blind faith in imagined authority figures you are…

> AI in its current form is a lot of automating assumption making. Imagine taking your assumption making powers, and extracting it into an unaccountable model that can be deployed at national or global scales.

Just say this part! This is what we could be talking about. It’s not a difficult mental model. But it’s just wrapped up in alarmist rhetoric that makes it hard for us to focus on the fact that the quote above is terrifying!!

Re: Only You Can Stop an AI Apocalypse

#88

Post by ShredKazoo »

jhbadger wrote: post #11

"Most AI researchers think good outcomes are more likely. This seems just blind faith, though" Or, you know, they might actually know something about the subject? On a similar note, most physicists don't think particle colliders will cause mini black holes which will swallow the Earth, but the fear gets repeated by non-physicists.

Part of what's going on here is that AI researchers are paid to tinker with systems and make them work, not think about these kind of big picture questions. A demonstration of this is that if you ask a big picture question a bit differently, you can get a very different answer. Example:

>...if you ask [AI researchers] for probabilities of things occurring in a fixed number of years, you get later estimates than if you ask for the number of years until a fixed probability will obtain. This looked very robust in 2016, and shows up again in the 2022 [human-level machine intelligence] data. Looking at just the people we asked for years, the aggregate forecast is 29 years, whereas it is 46 years for those asked for probabilities. (We haven’t checked in other data or for the bigger framing effect yet.)

https://www.lesswrong.com/posts/H6hMugfY3tDQGfqYL/what-do-ml...

Re: Only You Can Stop an AI Apocalypse

#89

Post by kamaal »

ShredKazoo wrote: post #83
nhod wrote: post #73

Actually, no, "only you" can't stop it. No one can. There was a book in the 80's called The Hundredth Monkey that posited that if only enough people opposed nuclear technology, there would be a magical critical mass of people after which it would become apparent to everyone in the whole world that nuclear technology was, you know, really bad and stuff, and it would all just go away. We know how that went. Like nuclea…

>We know how that went. A large majority of the world's countries have no nuclear weapons. Big swaths of the globe have been designated as "Nuclear-weapon-free zones": https://en.wikipedia.org/wiki/Nuclear-weapon-free_zone And we haven't had a nuclear apocalypse -- in part due to lots of work on things like weapons treaties, monitoring, control systems, etc. Throwing up our hands and giving up seems very premature. T…

>>And we haven't had a nuclear apocalypse -- in part due to lots of work on things like weapons treaties, monitoring, control systems, etc.

Mostly its because no body has been desperate enough to press the button yet. Even in cases with a power losing a war, they've had the attitude to 'live to fight another day'. So it hasn't come to pass.

The biggest fear is someday, a power, which is often known to get away with anything they do, will go one step too far thinking they won't be held accountable as usual, and the other side will say 'sorry, that was past, this is a little too much, we ran out of patience, here is a nuke move'.

Re: Only You Can Stop an AI Apocalypse

#90

Post by Animats »

SV_BubbleTime wrote: post #2

Good satire; thought it was real until I got to “We don’t know how our AI systems work, we don’t know what they can do,” I look forward to the stupid phase of mistakes with AI. Because it surely has to be better than the past decade of free money devoted almost exclusively to advertising optimization and tightening of controls.

“We don’t know how our AI systems work, we don’t know what they can do.”

That's not a joke. That's a real problem with the current versions of machine learning. You've got a huge collection of weights and no understanding of what they mean.

We've reached a strange place. Large language models have blown through the Turing Test. Yet this isn't AI. What we have is something that generates plausible-looking but not consistently correct text or images. Large language models have mechanized the Dunning-Kruger effect.

It's striking how plausible the output is. What this may demonstrate is that well-written text is mostly an average of a large body of text. This is similar to the discovery that if you average a large number of faces, you get a very good looking one. It's a discovery about the human perception system.

This may be a transient situation. Researchers are starting to figure out that inside those huge collections of weights, models sometimes emerge. Something that works like understanding or common sense may turn up in there. That may take a while.

Meanwhile, though, the business potential of automated blithering is going to result in this stuff being used for far too much.

Post reply on HN