Statement on AI Risk • Hacker News Board

Re: Statement on AI Risk

#711

Post by stevenhuang » Tue, May 30, 2023, 9:20 PM UTC

adamsmith143 wrote: post #515

chriskanan wrote: post #505

I have mixed feelings about this. This letter is much better than the earlier one. There is a growing percentage of legitimate AI researchers who think that AGI could occur relatively soon (including me). The concern is that it could be given objectives intentionally or unintentionally that could lead to an extinction event. Certainly LLMs alone aren't anything close to AGIs, but I think that autoregressive training…

>As far as a pandemic or nuclear war, though, I'd probably put it on more of the level of a K-T extinction event. Humans are doing some work on asteroid redirection, but I don't think it is a global priority. I think it's better to frame AI risks in terms of probability. I think the really bad case for humans is full extinction or something worse. What you should be doing is putting a probability distribution over th…

More appropriate is an expected value approach.

That is, despite it being a very low probability event, it may still be worth remediation due to the outsized negative value if the event does happen.

Many engineering disciplines incorporate safety factors to mitigate rare but catastrophic events for example.

If something is maximally bad, then it necessitates some deliberation on ways to avoid it, irrespective how unlikely seeming it may be.

Re: Statement on AI Risk

#712

Post by revelio » Tue, May 30, 2023, 9:21 PM UTC

hayst4ck wrote: post #653

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

Seth McFarland wrote a pretty great piece on Star Trek replicators and their relationship to the structure of society. The question it answers is "does the replicator allow for Star Trek's utopia, or does Star Trek's utopia allow for the replicator?" https://www.reddit.com/r/CuratedTumblr/comments/13tpq18/hear... It is very thought provoking, and very relevant.

It doesn't answer that, it can't because the replicator is fictional. McFarland just says he wrote an episode in which his answer is that replicators need communism, and then claims that you can't have a replicator in a capitalist system because evil conservatives, capitalists and conspiracy theorists would make strawman arguments against it.

Where is the thought provoking idea here? It's just an excuse to attack his imagined enemies. Indeed he dunks on conspiracy theorists whilst being one himself. In McFarland's world there would be a global conspiracy to suppress replicator technology, but it's a conspiracy of conspiracy theorists.

There's plenty of interesting analysis you could do on the concept of a replicator, but a Twitter thread like that isn't it. Really the argument is kind of nonsensical on its face because it assumes replicators would have a cost of zero to run or develop. In reality capitalist societies already invented various kinds of pseudo-replicators with computers being an obvious example, but this tech was ignored or suppressed by communist societies.

Re: Statement on AI Risk

#713

Post by nologic01 » Tue, May 30, 2023, 9:21 PM UTC

derbOac wrote: post #696

nologic01 wrote: post #677

This is a breathless, half-baked take on "AI Risk" that does not cast the esteemed signatories in a particularly glowing light. It is 2023. The use and abuse of people in the hands of information technology and automation has now a long history. "AI Risk" was not born yesterday. The first warning came as early as 1954 [1]. The Human Use of Human Beings is a book by Norbert Wiener, the founding thinker of cybernetics…

This topic clearly touches a nerve with the HN community, but I strongly agree with you. To be honest, I've been someone disappointed with the way AI/DL research has proceeded in the last several years and none of this really surprises me. From the beginning, this whole enterprise has been detached from basic computational and statistical theory. At some level this is fine — you don't need to understand everything yo…

> mathematicians, statisticians, and yes, philosophers and psychologists — than those who have been involved with it so far.

I think mathematicians and statisticians are hard to flummox but the risk with non-mathematically trained people such as philosophers and psychologists is that they can be sidetracked easily by vague and insinuating language that allows them to "fill-in" the gaps. They need an unbiased "interpreter" of what the tech actually does (or can do) and that might be hard to come by.

I would add political scientists and economists to the list. Not that I have particular faith in their track record solving any problem, but conceptually this is also their responsibility and privilege: technology reshapes society and the economy and we need to have a mature and open discussion about it.

Re: Statement on AI Risk

#714

Post by adamsmith143 » Tue, May 30, 2023, 9:22 PM UTC

cmilton wrote: post #546

adamsmith143 wrote: post #522

Earlier quoted context omitted.

If AI improves by 0.0001% per year on your favorite intelligence metric there will eventually be a point where it surpasses human performance and another point where it surpasses all humans combined on that metric. There is danger in that scenario.

Assuming that timeline, can we agree that we have some time (years?) to hash this out further before succumbing to the ideals of a select few?

The problem is that even with N years until we reach that point it seems likely that it would take 2*N years to build the proper safety mechanisms because at least currently capabilities research is racing far ahead of safety research. Of course we have no way to know how big N really is and recent results like GPT-4, Llama, Gato, etc. have shifted peoples timelines significantly. So even if 5 years ago people like Geoff Hinton though this might be 30-50 years away there are now believable arguments to make that it might be more like 3-10 years.

Re: Statement on AI Risk

#715

Post by revelio » Tue, May 30, 2023, 9:26 PM UTC

arisAlexis wrote: post #701

revelio wrote: post #585

Earlier quoted context omitted.

Academic research involves large components of marketing. That's why they grumble so much about the time required in the grant applications process and other fund seeking effort. It's why they so frequently write books, appear in newspaper articles and on TV. It's why universities have press relations teams.

Again since these are almost the top cream of all ai researchers there is a global conspiracy to scare the public right? Has it occurred to you what happens if you are wrong, like 10% chance you are wrong? Well it's written in the declaration.

No, lots of important AI researchers are missing and many of the signatories have no relevant AI research experience. As for being the cats whiskers in developing neural architecture or whatever, so what? It gives them no particular insight into AI risk. Their papers are mostly public, remember.

> Has it occurred to you what happens if you are wrong?

Has it occurred to you what happens if YOU are wrong? AI risk is theoretical, vague and most arguments for it are weak. The risk of bad law making is very real, has crushed whole societies before and could easily cripple technological progress for decades or even centuries.

IOW the risk posed by AI risk advocates is far higher than the risk posed by AI.

Re: Statement on AI Risk

#716

Post by [deleted] » Tue, May 30, 2023, 9:26 PM UTC

MrScruff wrote: post #188

I find it quite extraordinary how many on here are dismissing that there is any risk at all. I also find statements like Yann LeCunn's that "The most common reaction by AI researchers to these prophecies of doom is face palming." to be lacking in awareness. "Experts disagree on risk of extinction" isn't quite as reassuring as he thinks it is. The reality is, despite the opinions of the armchair quarterbacks commentin…

[deleted]

Re: Statement on AI Risk

#717

Post by YeGoblynQueenne » Tue, May 30, 2023, 9:27 PM UTC

mitthrowaway2 wrote: post #389

YeGoblynQueenne wrote: post #355

Earlier quoted context omitted.

Do you mean you're not aware of the arguments in favour of stopping climate change?

No need; like I said, I'm all in favour of fighting climate change. I view it as an existential risk to humanity on the ~200 year timescale, and it should be a high priority. I'm particularly concerned about the impacts on ocean chemistry. But if you're going to suggest that a Statement on AI risk should mention climate change but not AI risk, because it's a "fantasy", then... well, I'd expect some kind of supporting…

There's no false dichotomy, but a very real one. One problem is current and pressing, the other is a fantasy. I don't need to support that with any argument: the non-existence of superintelligent AGI is not disputed, nor is any of the people crying doom claim that they, or anyone else, know how to create one. It's an imaginary risk.

Re: Statement on AI Risk

#718

Post by shrimpx » Tue, May 30, 2023, 9:29 PM UTC

toth wrote: post #710

shrimpx wrote: post #680

Earlier quoted context omitted.

> academics Academics get paid (and compete hardcore) for creating status and prominence for themselves and their affiliations. Suddenly 'signatory on XYZ open letter' is an attention source and status symbol. Not saying this is absolutely the case, but academics putting their name on something surrounded by hype isn't the ethical check you make it out to be.

This a letter anyone can sign. As someone pointed out Grimes is one of the signatories. You can sign it yourself. Hinton, Bengio, Norvig and Russell are most definitely not getting prestige from signing it. The letter itself is getting prestige from them having signed it.

Nah, they're getting visibility from the topic of 'AI risk'. I don't know who those people are but this AI risk hype is everywhere I look including in congressional hearings.

Re: Statement on AI Risk

#719

Post by adamsmith143 » Tue, May 30, 2023, 9:29 PM UTC

stevenhuang wrote: post #711

adamsmith143 wrote: post #515

Earlier quoted context omitted.

>As far as a pandemic or nuclear war, though, I'd probably put it on more of the level of a K-T extinction event. Humans are doing some work on asteroid redirection, but I don't think it is a global priority. I think it's better to frame AI risks in terms of probability. I think the really bad case for humans is full extinction or something worse. What you should be doing is putting a probability distribution over th…

More appropriate is an expected value approach. That is, despite it being a very low probability event, it may still be worth remediation due to the outsized negative value if the event does happen. Many engineering disciplines incorporate safety factors to mitigate rare but catastrophic events for example. If something is maximally bad, then it necessitates some deliberation on ways to avoid it, irrespective how unl…

Exactly. Taken to the limit if you extrapolate how many future human lives could be extinguished by a runaway AI you get extremely unsettling answers. Like the expected value of a .01% change of extinction from AI might be Trillions of quality Human lives. (This could in fact be on the very very conservative side, e.g. Nick Bostrom has speculated that there could be 10^35 human lives to be lived in the far future which is itself a conservative estimate). With these numbers even setting AI risk to be absurdly low, say 1/10^20, we might still expect to lose 10 billion lives. (I'd argue even the most optimistic person in the world couldn't assign a probability that low) So the stakes are extraordinarily high.

https://globalprioritiesinstitute.org/wp-content/uploads/Tob...

Re: Statement on AI Risk

#720

Post by hollerith » Tue, May 30, 2023, 9:31 PM UTC

deltaninenine wrote: post #280

shafyy wrote: post #32

The issue I take with these kind of "AI safety" organizations is that they focus on the wrong aspects of AI safety. Specifically, they run this narrative that AI will make us humans go extinct. This is not a real risk today. Real risks are more in the category of systemic racism and sexism, deep fakes, over reliance on AI etc. But of course, "AI will humans extinct" is much sexier and collects clicks. Therefore, the…

Don't characterize the public as that stupid. The current risks of AI are startling clear to a layman. The extinction level even is more far fetched to a layman. You are the public and your viewpoint is aligned with the public. Nobody is thinking extinction level event.

Extinction is exactly what this submission is about.

Here is the full text of the statement: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

By "extinction", the signatories mean extinction of the human species.