Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

International Scientific Report on the Safety of Advanced AI [pdf]

assets.publishing.service.gov.uk

Post reply on HN
Print view View original

21–30 of 66 posts

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#21

Post by kbenson »

ben_w wrote: post #19
hiAndrewQuinn wrote: post #16

Recursively self-improving AI, of the kind Nick Bostrom outlined in detail way back in his 2014 book Superintelligence and Dr. Omohundro outlined in brief in [1], is the only kind which poses a true existential threat. I don't get out of bed for people worrying about anything less when it comes to 'AI safety'. On the topic: One potentially effective approach to stopping recursive self-improving AI from being develope…

> is the only kind which poses a true existential threat You don't accept the possibility that a non-improving tool of an AI system that is fixed at the level of "just got a PhD in everything" by reading all the research papers on arxiv, might possibly be advanced enough for a Jim Jones type figure to design and create a humanity-ending plague because they believe in bringing about the end times?

Wouldn't there be 100x more of the same capability looking for threats and trying to head them off? A very advanced tool is still just a tool, and subject to countermeasures.

I can see why countries would want to regulate it, but personally I think it's a distinctly different category than what the GP comment was talking about.

There is no stopping a singularity level event after it's begun, at least not by any process where people play a role.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#22

Post by consumer451 »

I understand that this report is not really about AGI, but I would like to again raise my main concern: I am much more worried about the implications of the real threat of dumb humans using dumb "AI" in the near-term, than I am about the theoretical threat of AGI.

Example:

https://news.ycombinator.com/item?id=39944826

https://news.ycombinator.com/item?id=39918245

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#23

Post by mrshadowgoose »

hiAndrewQuinn wrote: post #16

Recursively self-improving AI, of the kind Nick Bostrom outlined in detail way back in his 2014 book Superintelligence and Dr. Omohundro outlined in brief in [1], is the only kind which poses a true existential threat. I don't get out of bed for people worrying about anything less when it comes to 'AI safety'. On the topic: One potentially effective approach to stopping recursive self-improving AI from being develope…

In your mind, what will our world look like once AGI is achieved, and that technology will likely be exclusively in the hands of governments and large corporations? What guarantees do we have that it will be used for the benefit of the common person? What will happen to the large swathes of the human population that will not only be permanently economically useless, but worse than useless? They'll need to be fed, clothed, housed and entertained with the only things they provide back being their opinions and complaints.

I literally couldn't care less about recursively-improving superintelligence. AGI in the hands of the rich and powerful is already a nightmare scenario.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#24

Post by api »

joebob42 wrote: post #2

How would they know? We don't have general AI but it's written as if they already know what it will be, how safe it will be, etc. I think it's an important topic to discuss and consider, but this seems to be speaking with more knowledge and authority than seems reasonable to me.

You’ve hit on a giant thing that bothers me about this discourse: endless rationalistic discourse about systems and phenomena that we have no experience with at all, not even analogous experience.

This is not like the atomic bomb. We had tons of experience with big bombs. We just knew atom bombs if they worked could make orders of magnitude larger booms. The implications if real big bombs could be reasoned about with some basis in reality.

It wasn’t reasoning about wholly unknown types of things that no human being has ever encountered or interacted with.

This is like a panel on protocols for extraterrestrial contact. It’d be fine to do that kind of exercise academically but these people are talking about passing actual laws and regulations on the basis of reasoning in a vacuum.

We are going to end up with laws and regulations that will be simultaneously too restrictive to human endeavor and ineffective at preventing negative outcomes if this stuff ever manifests for real.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#25

Post by ninetyninenine »

warkdarrior wrote: post #7
ninetyninenine wrote: post #3

The biggest risk of ai can be simply characterized by one concept: This very report could have been generated by AI. We can’t fully tell.

So what if it was generated by AI? Does that invalidate its contents in any way?

No it does not.

This is the scary part.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#26

Post by hollerith »

wslh wrote: post #18

Clearly AI is unstoppable as it is math. I don't get how politicians or intelectuals continue to argue without understanding that simple truth. I don't know if AI will be comparable to human or organizational intelligence but know that is unstoppable. Just ranting but a potential way of defending about it is taking an approach similar to cryptography for quantum computers: think harder if we can excel on something ev…

>Clearly AI is unstoppable as it is math.

I don't understand this argument. It takes years to become competent at the math needed for AI. If stopping AI is important enough, society will make teaching, learning and disseminating writings about that math illegal. After that, almost no one is going to invest the years needed to master the math because it no longer helps anyone advance in their careers, and the vast majority of the effect of math on society is caused by people who learned the math because they expected it to advance their career.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#27

Post by zarzavat »

thegrim33 wrote: post #5

"At a time of unprecedented progress in AI development, this first publication restricts its focus to a type of AI that has advanced particularly rapidly in recent years: General-purpose AI, or AI that can perform a wide variety of tasks" Is "general-purpose AI" supposed to be something different from "AGI" or from "General Artificial Intelligence"? Or is it yet another ambiguous ill-defined term that means something…

AlphaFold is special-purpose AI. GPT-4o is general-purpose AI. Seems clear to me.

“AGI” is a specific term of art that is a subset of general-purpose AI.

AGI would have the capability to learn and reason at the same level as the most capable humans, and therefore perform any intellectual task that a human can.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#28

Post by FeepingCreature »

api wrote: post #24
joebob42 wrote: post #2

How would they know? We don't have general AI but it's written as if they already know what it will be, how safe it will be, etc. I think it's an important topic to discuss and consider, but this seems to be speaking with more knowledge and authority than seems reasonable to me.

You’ve hit on a giant thing that bothers me about this discourse: endless rationalistic discourse about systems and phenomena that we have no experience with at all, not even analogous experience. This is not like the atomic bomb. We had tons of experience with big bombs. We just knew atom bombs if they worked could make orders of magnitude larger booms. The implications if real big bombs could be reasoned about with…

I mean, we have an entire biosphere, us included, for samples of entities of varying intelligence.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#29

Post by FeepingCreature »

GPerson wrote: post #12
krisoft wrote: post #9

Earlier quoted context omitted.

> It's funny that they claim that "general purpose AI" has "advanced particularly rapidly" even though they didn't, nor can't, define what it even is. I'm confused about what you are missing. You are quoting their definition. "General-purpose AI, or AI that can perform a wide variety of tasks". That's their definition. You might not like it but that is a definition.

A corporation, a question and answer website, a Minecraft game, and basically any sufficiently complex system could all be General-purpose AIs by that definition though. Having an overly general and useless definition is not better than having no definition in my opinion, so I see where the OP is coming from.

I have seen people call corporations AIs. This is not obviously wrong.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#30

Post by api »

FeepingCreature wrote: post #28
api wrote: post #24

Earlier quoted context omitted.

You’ve hit on a giant thing that bothers me about this discourse: endless rationalistic discourse about systems and phenomena that we have no experience with at all, not even analogous experience. This is not like the atomic bomb. We had tons of experience with big bombs. We just knew atom bombs if they worked could make orders of magnitude larger booms. The implications if real big bombs could be reasoned about with…

I mean, we have an entire biosphere, us included, for samples of entities of varying intelligence.

There are so many differences here vs organisms shaped by evolution and involved in a food web with each other. This is much closer to space aliens or beings from another dimension.

If there are huge risks here they are probably not the ones we are worried about.

Personally one of my biggest worries with both sentient AI and aliens is how humans might react and what we might do to each other or ourselves out of fear and paranoia.

Post reply on HN