Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

International Scientific Report on the Safety of Advanced AI [pdf]

assets.publishing.service.gov.uk

Post reply on HN
Print view View original

31–40 of 66 posts

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#31

Post by wslh »

hollerith wrote: post #26
wslh wrote: post #18

Clearly AI is unstoppable as it is math. I don't get how politicians or intelectuals continue to argue without understanding that simple truth. I don't know if AI will be comparable to human or organizational intelligence but know that is unstoppable. Just ranting but a potential way of defending about it is taking an approach similar to cryptography for quantum computers: think harder if we can excel on something ev…

>Clearly AI is unstoppable as it is math. I don't understand this argument. It takes years to become competent at the math needed for AI. If stopping AI is important enough, society will make teaching, learning and disseminating writings about that math illegal. After that, almost no one is going to invest the years needed to master the math because it no longer helps anyone advance in their careers, and the vast maj…

It is very simple: powerful governments tried to stop cryptography, we know what happened. Also governments tried to prohibit alcohol, etc. it does not work. You can get them even in places such as Saudi Arabia. Are they expensive? For sure, but when it is about science that you can run in your own computers nothing can stop it. Will they put a Clipper chip?

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#32

Post by wyldfire »

hollerith wrote: post #26
wslh wrote: post #18

Clearly AI is unstoppable as it is math. I don't get how politicians or intelectuals continue to argue without understanding that simple truth. I don't know if AI will be comparable to human or organizational intelligence but know that is unstoppable. Just ranting but a potential way of defending about it is taking an approach similar to cryptography for quantum computers: think harder if we can excel on something ev…

>Clearly AI is unstoppable as it is math. I don't understand this argument. It takes years to become competent at the math needed for AI. If stopping AI is important enough, society will make teaching, learning and disseminating writings about that math illegal. After that, almost no one is going to invest the years needed to master the math because it no longer helps anyone advance in their careers, and the vast maj…

> society will make teaching, learning and publishing about that math illegal

If there were ever a candidate for Poe's Law comment of the year, this comment on HN would be it.

So much literature depicts just such a dystopia where the technology unleashes humanity's worst and they decide to prevent education in order to avoid the fate of the previously fallen empire.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#33

Post by boesboes »

joebob42 wrote: post #2

How would they know? We don't have general AI but it's written as if they already know what it will be, how safe it will be, etc. I think it's an important topic to discuss and consider, but this seems to be speaking with more knowledge and authority than seems reasonable to me.

general purpose AI != AGI

They just means 'not trained for exactly one tasks', i.e LLMs and such and not AlhpaFold.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#34

Post by blackeyeblitzar »

I feel like a lot of the arguments listed in “4.1.2 Disinformation and manipulation of public opinion” apply in a non AI world. In social media, sites like Hacker News are rare. In most places you’ll see comments that are (purposely) sharing a subset of the whole truth in order to push a certain opinion. From my perspective disinformation and manipulation of public opinion already happens “at scale”. Most social media users belong to one political side or the other, and there are lots of them, and almost all of them are either factually incorrect or lack nuance or an understanding of their opponents’ view.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#35

Post by johndough »

hollerith wrote: post #26
wslh wrote: post #18

Clearly AI is unstoppable as it is math. I don't get how politicians or intelectuals continue to argue without understanding that simple truth. I don't know if AI will be comparable to human or organizational intelligence but know that is unstoppable. Just ranting but a potential way of defending about it is taking an approach similar to cryptography for quantum computers: think harder if we can excel on something ev…

>Clearly AI is unstoppable as it is math. I don't understand this argument. It takes years to become competent at the math needed for AI. If stopping AI is important enough, society will make teaching, learning and disseminating writings about that math illegal. After that, almost no one is going to invest the years needed to master the math because it no longer helps anyone advance in their careers, and the vast maj…

> It takes years to become competent at the math needed for AI

(Assuming that "AI" refers to large language models)

The best open source LLM fits in less than 300 lines of code and consists mostly of matrix multiplications. https://github.com/meta-llama/llama3/blob/main/llama/model.p...

Anyone with a basic grasp of linear algebra can probably learn to understand it in a week. Here is a video playlist by former Stanford professor and OpenAI employee Andrej Karpathy which should cover most of it (less than 16 hours total): https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxb...

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#36

Post by jprete »

GPerson wrote: post #12
krisoft wrote: post #9

Earlier quoted context omitted.

> It's funny that they claim that "general purpose AI" has "advanced particularly rapidly" even though they didn't, nor can't, define what it even is. I'm confused about what you are missing. You are quoting their definition. "General-purpose AI, or AI that can perform a wide variety of tasks". That's their definition. You might not like it but that is a definition.

A corporation, a question and answer website, a Minecraft game, and basically any sufficiently complex system could all be General-purpose AIs by that definition though. Having an overly general and useless definition is not better than having no definition in my opinion, so I see where the OP is coming from.

It's general-purpose AI as AI is commonly understood, which is probably specific and useful enough for such a report.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#37

Post by impossiblefork »

ben_w wrote: post #19
hiAndrewQuinn wrote: post #16

Recursively self-improving AI, of the kind Nick Bostrom outlined in detail way back in his 2014 book Superintelligence and Dr. Omohundro outlined in brief in [1], is the only kind which poses a true existential threat. I don't get out of bed for people worrying about anything less when it comes to 'AI safety'. On the topic: One potentially effective approach to stopping recursive self-improving AI from being develope…

> is the only kind which poses a true existential threat You don't accept the possibility that a non-improving tool of an AI system that is fixed at the level of "just got a PhD in everything" by reading all the research papers on arxiv, might possibly be advanced enough for a Jim Jones type figure to design and create a humanity-ending plague because they believe in bringing about the end times?

I believe that plagues are really easy to make, and that making them is already accessible to most PhDs in biomedicine, many PhD students and some particularly talented high school students.

Most of skills are I believe, laboratory and biology experiment debugging skills rather than something like deductive intelligence.

The easiest things aren't actually 'plagues' as such, and the high school accessible approaches would require access to being able to order things from DNA synthesis labs. I think it's accessible to an incredible number of people, basically everyone I know who does biology. None of them would ever think about doing this though, and if I asked them they would probably not know how to do it, because they'd never direct their thought in that direction.

I think the real concern with an AI system that is like someone with a PhD in everything should rather be that it'll be really hard to get a job if that kind of thing is available. It'll give enormous power to land and capital owners. That is something which is really dangerous enough though.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#38

Post by [deleted] »

euroderf wrote: post #13
warkdarrior wrote: post #7

Earlier quoted context omitted.

So what if it was generated by AI? Does that invalidate its contents in any way?

Interesting point. Are we already at a state where an A.I. could respond to email, phone calls, and even video calls in a convincing way ?

[deleted]

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#39

Post by hollerith »

wslh wrote: post #31
hollerith wrote: post #26

Earlier quoted context omitted.

>Clearly AI is unstoppable as it is math. I don't understand this argument. It takes years to become competent at the math needed for AI. If stopping AI is important enough, society will make teaching, learning and disseminating writings about that math illegal. After that, almost no one is going to invest the years needed to master the math because it no longer helps anyone advance in their careers, and the vast maj…

It is very simple: powerful governments tried to stop cryptography, we know what happened. Also governments tried to prohibit alcohol, etc. it does not work. You can get them even in places such as Saudi Arabia. Are they expensive? For sure, but when it is about science that you can run in your own computers nothing can stop it. Will they put a Clipper chip?

The difference between "stop AI" and "stop cryptography" is that those of us who want to stop AI want to stop AI models from becoming more powerful by stopping future mathematical discoveries in the field. In contrast, the people trying to stop cryptography were trying to stop the dissemination of math that had already been discovered and understood well enough to have been productized in the form of software.

Western society made a decision in the 1970s to stop human germ-line engineering and cloning of humans, and so far those things have indeed been stopped not only in the West, but worldwide. They've been stopped because no one currently knows of an effective way to, e.g., add a new gene to a human embryo. I mean that (unlike the situation in cryptography) there is no "readily-available solution" that enables it to be done without a lengthy and expensive research effort. And the reason for that lack of availability of a "readily-available solution" is the fact that no young scientists or apprentice scientists have been working on such a solution -- because every scientist and apprentice scientist understood and understands that spending any significant time on it would be a bad career move.

Those of us who want to stop AI don't care you you run LLama on your 4090 at home. We don't even care if ChatGPT, etc, remain available to everyone. We don't care because LLama and ChatGPT have been deployed long enough and in enough diverse situations that if any of them were dangerous, the harm would have occurred by now. We do want to stop people from devoting their careers to looking for new insights that would enable more powerful AI models.

Re: International Scientific Report on the Safety of Advanced AI [pdf]

#40

Post by semi-extrinsic »

ninetyninenine wrote: post #25
warkdarrior wrote: post #7

Earlier quoted context omitted.

So what if it was generated by AI? Does that invalidate its contents in any way?

No it does not. This is the scary part.

Do you mean scary as in "it is scary that apparently intelligent and sane people are wasting time and money on producing documents that consist purely of meaningless fluff"? If so, I agree.
Post reply on HN