Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

581–590 of 964 posts

Re: Statement on AI Risk

#581

Post by SpaceManNabs »

brookst wrote: post #252
SpaceManNabs wrote: post #237

Earlier quoted context omitted.

> They can’t point to an existing system that poses existential risk, because it doesn’t exist. There are judges using automated decision systems to excuse away decisions that send people back to jail for recidivism purposes. These systems are just enforcing societal biases at scale. It is clear that we are ready to acquiesce control to AI systems without much care to any extra ethical considerations.

I think you just said that the problem is systemic in our judicial system, and that AI has nothing to do with it.

AI is the tool that provides "objective to truth" that enables such behavior. It is definite unique in its depth, scale, and implications.

Re: Statement on AI Risk

#582

Post by fauigerzigerk »

duvenaud wrote: post #475
fauigerzigerk wrote: post #384

Earlier quoted context omitted.

>What abilities do humans have that AIs will never have? I think the question is what abilities and level of organisation machines would have to acquire in order to outcompete entire human societies in the quest for power. That's a far higher bar than outcompeting all individual humans at all cognitive tasks.

Good point. Although in some ways it's a lower bar, since agents that can control organizations can delegate most of the difficult tasks. Most rulers don't invent their own societies from scratch, they simply co-opt existing power structures or political movements. El Chapo can run a large, powerful organization from jail.

That would require a high degree of integration into human society though, which makes it seem very unlikely that AIs would doggedly pursue a common goal that is completely unaligned with human societies.

Extinction or submission of human society via that route could only work if there was a species of AI that would agree to execute a secret plan to overcome the rule of humanity. That seems extremely implausible to me.

How would many different AIs, initially under the control of many different organisations and people, agree on anything? How would some of them secretly infiltrate and leverage human power structures without facing opposition from other equally capable AIs, possibly controlled by humans?

I think it's more plausible to assume a huge diversity of AIs, well integrated into human societies, playing a role in combined human-AI power struggles rather than a species v species scenario.

Re: Statement on AI Risk

#584

Post by revelio »

kbash9 wrote: post #545
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

The risks are definitely real. Just look at the number of smart individuals speaking out about this. The argument that anybody can build this in their basement is not accurate at the moment - you need a large cluster of GPUs to be able to come close to state of the art LLMs (e.g. GPT4). Sam Altman's suggestion of having an IAEA [ https://www.iaea.org/ ] like global regulatory authority seems like the best course of a…

> The risks are definitely real. Just look at the number of smart individuals speaking out about this.

In our society smart people are strongly incentivized to invent bizarre risks in order to reap fame and glory. There is no social penalty if those risks never materialize, turn out to be exaggerated or based on fundamental misunderstanding. They just shrug and say, well, better safe than sorry, and everyone lets them off.

So you can't decide the risks are real just by counting "smart people" (deeply debatable how that's defined anyway). You have to look at their arguments.

Re: Statement on AI Risk

#585

Post by revelio »

arisAlexis wrote: post #453
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

All academics and researchers from different parts of the world reek marketing? Conspiracy theorists are strong

Academic research involves large components of marketing. That's why they grumble so much about the time required in the grant applications process and other fund seeking effort. It's why they so frequently write books, appear in newspaper articles and on TV. It's why universities have press relations teams.

Re: Statement on AI Risk

#586

Post by olddustytrail »

Der_Einzige wrote: post #327
kalkin wrote: post #214

Earlier quoted context omitted.

"nuclear weapons are no big deal actually" is just a wild place to get as a result of arguing against AI risk. Although I guess Eliezer Yudkowsky would agree! (On grounds that nukes won't kill literally everyone while AI will, but still.)

Nuclear weapons are uniquely good. Turns out you have to put guns to the collective temples of humanity for them to realize that pulling the trigger is a bad idea.

Please Google "Blackadder how did the war start video" and watch.

Re: Statement on AI Risk

#587

Post by boringuser2 »

ever1337 wrote: post #569
boringuser2 wrote: post #393

Earlier quoted context omitted.

Very Dunning-Kruger post right here.

Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.

You're a priori writing off my comment as fruitless because of your emotions and not because you actually have given it deep thought and carefully reached the conclusion that social feedback is somehow bad.

Also, the notion that "people's work" is inherently worthy of respect is just nonsensical. I do shoddy work all the time. Hell, you just casually dismissed my internet comment work as shallow and told me not to do it. Please don't post a shallow dismissal of my work.

Don't you think that this is all a bit anti-intellectual?

Re: Statement on AI Risk

#588

Post by NateEag »

Mizza wrote: post #160
NateEag wrote: post #86

Earlier quoted context omitted.

If you assume without evidence that recursively self-improving intelligence massively by thinking is possible, then it follows that severe existential risk from AI is plausible. If a software system did develop independent thought, then found a way to become, say, ten times smarter than a human, then yeah - whatever goals it set out to achieve, it probably could. It can make a decent amount of money by taking freelan…

The idea of a superintelligence becoming a bond villain via freelance software jobs (or, let's be honest, OnlyFans scamming) is not something I consider an existential threat. I can't find it anything other than laughable. It's like you've looked at the Fermi paradox and decided we need Congress to immediately invest in anti-alien defense forces. It's super-intelligent and it's a super-hacker and it's a super-crimina…

> The idea of a superintelligence becoming a bond villain via freelance software jobs (or, let's be honest, OnlyFans scamming) is not something I consider an existential threat. I can't find it anything other than laughable.

Conservative evangelical Christians find evolution laughable.

Finding something laughable is not a good reason to dismiss it as impossible. Indeed, it's probably a good reason to think "What am I so dangerously certain of that I find contradictory ideas comical?"

> Meanwhile, I work with LLMs every day and can only get them to print properly formatted JSON "some" of the time. Get real.

I don't think the current generation of LLMs is anything like AGI, nor an existential risk.

That doesn't mean it's impossible for some future software system to present an existential risk.

Re: Statement on AI Risk

#589

Post by boringuser2 »

dopidopHN wrote: post #423
boringuser2 wrote: post #413

A lot of people in this thread seem to be suffering from a lack of compute. The idea that an AI can't be dangerous because it is a incorporeal entity trapped in electricity is particularly dumb. This is literally how your brain works. You didn't build your house. You farmed the work out using leverage to people with the skills and materials. Your leverage was meager wealth generated by a loan. The leverage of a super…

I struggle to find description of how that would look like in non fiction sources. But your take and analogies are the best strain of ideas I heard so far… How would that would look ? An AGI hidding it’s state, and effect on the real world thought the internet. It’s not like we did not build thousand of venues for that thought various API. Or just task rabbits.

The actions of a malicious AI cannot be simulated, because this would require an inferior intellect predicting a superior intellect. It's P versus NP.

The point to make is that it is trivial to imagine an AI wielding power even within the confines of human-defined intellect. For example, depictions of AI in fiction typically present as a really smart human that can solve tasks instantly. Obviously, this is still within the realm of failing to predict a fundamentally superior intellect, but it still presents the terrifying scenario that simply doing exceptional human-level tasks very quickly is existentially unsettling.

Mere leverage has sufficient explanatory power to explain the efficacy of an intelligent artificial agent, let alone getting into speculation about network security, hacking, etc.

Re: Statement on AI Risk

#590

Post by klibertp »

sebzim4500 wrote: post #80
vesinisa wrote: post #53

Earlier quoted context omitted.

Yeah, like I fail to see how would an AI even cause human extinction? Through some Terminator style man-robot warfare? But the only orgnizations that would seem capable of building such killer robots are governments that already possess the capacity to extinguish the entire human race with thermonuclear weapons - and at a considerably lower R&D budget for that end. It seems like hysteria / clever marketing for AI pro…

The standard example is that it would engineer a virus but that's probably a lack of imagination. There may be more reliable ways of wiping out humanity that we can't think of. I think speculation on the methods is pretty pointless, if a superintelligent AI is trying to kill us we're probably going to die, the focus should be on avoiding this situation. Or providing a sufficiently convincing argument for why that won…

Or why it should happen...
Post reply on HN