Statement on AI Risk • Hacker News Board

Re: Statement on AI Risk

#511

Post by drvdevd » Tue, May 30, 2023, 6:05 PM UTC

ChatGTP wrote: post #89

blueblimp wrote: post #35

Earlier quoted context omitted.

There is a way, in my opinion: distribute AI widely and give it a diversity of values, so that any one AI attempting takeover (or being misused) is opposed by the others. This is best achieved by having both open source and a competitive market of many companies with their own proprietary models.

How do you give "AI" a diversity of values?

By driving down the costs of training and inference, and then encouraging experiments. For LLMs, QLoRA is arguably a great step in this direction.

Re: Statement on AI Risk

#512

Post by myrmidon » Tue, May 30, 2023, 6:06 PM UTC

silverlake wrote: post #131

wongarsu wrote: post #54

Earlier quoted context omitted.

A good way to categorize risk is look at both likelihood and severity of consequences. The most visible issues today (racism, deep fakes, over reliance) are almost certain to occur, but also for the most part have relatively minor consequences (mostly making things that are already happening worse). "Advanced AI will make humans extinct" is much less likely but has catastrophic consequences. Focusing on the catastrop…

This doesn’t work either. The consequence of extinction is infinity (to humans). Likelihood * infinity = infinity. So by hand-waving at a catastrophic sci-fi scenario they can demand we heed their demands, whatever that is.

If you want to prevent this, you simply have to show that the probability for that extinction scenario is lower than the baseline where we start to care.

Lets take "big asteroid impact" as baseline because that is a credible risk and somewhat feasible to quantify: Probability is somewhere under 1 in a million over a human lifetime, and we barely care (=> we do care enough to pay for probe missions investigating possible mitigations!).

So the following requirements:

1) Humanity creates one or more AI agents with strictly superhuman cognitive abilities within the century

2) AI acquires power/means to effect human extinction

3) AI decides against coexistence with humans

Only need 1% probability each to exceed that probability bound. And especially 1) and 3) seem significantly more likely than 1% to me, so the conclusion would be that we should worry about AI extinction risks...

Re: Statement on AI Risk

#513

Post by londons_explore » Tue, May 30, 2023, 6:06 PM UTC

toth wrote: post #490

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I think you are wrong. The risks are real and, while I am sure OpenAI and others will position themselves to take advantage of regulations that emerge, I believe that the CEOs are doing this at least in part because they believe this. If this was all about regulatory capture and marketing, why would Hinton, Bengio and all the other academics have signed the letter as well? Their only motivation is concern about the r…

The risks are real, but I don't think regulations will mitigate them. It's almost impossible to regulate something you can develop in a basement anywhere in the world.

The real risks are being used to try to built a regulatory moat, for a young industry who famously has no moat.

Re: Statement on AI Risk

#514

Post by EamonnMR » Tue, May 30, 2023, 6:06 PM UTC

mitthrowaway2 wrote: post #339

sf4lifer wrote: post #139

LLMs are just text prediction. I don't see the linear path from LLM to AGI. Was there similar hysteria when the calculator or PC first came out?

The statement isn't about LLMs. It doesn't refer to them even once.

But if LLMs hadn't captured the popular imagination I doubt this would have been written this year and gotten the attention of enough prominent signatories to frontpage on HN.

Re: Statement on AI Risk

#515

Post by adamsmith143 » Tue, May 30, 2023, 6:07 PM UTC

chriskanan wrote: post #505

I have mixed feelings about this. This letter is much better than the earlier one. There is a growing percentage of legitimate AI researchers who think that AGI could occur relatively soon (including me). The concern is that it could be given objectives intentionally or unintentionally that could lead to an extinction event. Certainly LLMs alone aren't anything close to AGIs, but I think that autoregressive training…

>As far as a pandemic or nuclear war, though, I'd probably put it on more of the level of a K-T extinction event. Humans are doing some work on asteroid redirection, but I don't think it is a global priority.

I think it's better to frame AI risks in terms of probability. I think the really bad case for humans is full extinction or something worse. What you should be doing is putting a probability distribution over that possibility instead of trying to guess how bad it could be, it's safe to assume it would be maximally bad.

Re: Statement on AI Risk

#516

Post by EamonnMR » Tue, May 30, 2023, 6:07 PM UTC

1970-01-01 wrote: post #323

>The succinct statement below... How does AI morph from an existential crisis in software development into a doomsday mechanism? It feels like all this noise stems from ChatGPT. And the end result is going to be a "DANGER! ARTIFICIAL INTELLIGENT SOFTWARE IN USE" sticker on my next iPhone.

Please make that sticker, I will buy it.

Re: Statement on AI Risk

#517

Post by staunton » Tue, May 30, 2023, 6:09 PM UTC

goolulusaurs wrote: post #467

jackbrookes wrote: post #445

Earlier quoted context omitted.

Of course every one has been wrong. If they were right, you wouldn't be here talking about it. It shouldn't be surprising that everyone has been wrong before

Consider two different scenarios: 1) Throughout history many people have predicted the world would soon end, and the world did not in fact end. 2) Throughout history no one predicted the world would soon end, and the world did not in fact end. The fact that the real world is aligned with scenario 1 is more an indication that there exists a pervasive human cognitive bias to think that the world is going to end, which…

That argument is still invalid because in scenario 2 we would not be having this discussion. No conclusions can be drawn from such past discourse about the likelihood of definite and complete extinction.

Not that, I hope, anyone expected a strong argument to be had there. It seems reasonably certain to me that humanity will go extinct one way or another eventually. That is also not a good argument in this situation.

Re: Statement on AI Risk

#518

Post by carapace » Tue, May 30, 2023, 6:11 PM UTC

holmesworcester wrote: post #478

dopamean wrote: post #431

Earlier quoted context omitted.

I agree that nothing about the statement makes me think the risks are real however I disagree that if the risks are real these companies would stop working on their product. I think more realistically they'd shut up about the risk and downplay it a lot. Much like the oil industry did wrt climate change going back to the 70's.

Yes, this! It is extremely rare for companies or their senior staff to beg for regulation this far in advance of any big push by legislators or the public. The interpretation that this is some 3-D chess on the companies' part is a huge violation of Occam's Razor.

Ockham's Razor doesn't apply in adversarial situations.

- - - -

I think the primary risk these folks are worried about is loss of control. And in turn, that's because they're all people for whom the system has more-or-less worked.

Poor people are worried the risk that the rich will keep the economic windfall to themselves and not share it.

Re: Statement on AI Risk

#519

Post by adamsmith143 » Tue, May 30, 2023, 6:11 PM UTC

shafyy wrote: post #32

The issue I take with these kind of "AI safety" organizations is that they focus on the wrong aspects of AI safety. Specifically, they run this narrative that AI will make us humans go extinct. This is not a real risk today. Real risks are more in the category of systemic racism and sexism, deep fakes, over reliance on AI etc. But of course, "AI will humans extinct" is much sexier and collects clicks. Therefore, the…

>Real risks are more in the category of systemic racism and sexism, deep fakes, over reliance on AI etc.

This is a really bad take and risks missing the forest for the trees in a major way. The risks of today pale in comparison to the risks of tomorrow in this case. It's like being worried about birds dying in wind turbines while the world ecosystem collapses due to climate change. The larger risk is further away in time but far more important.

Theres a real risk that people get fooled by this idea that LLMs saying bad words is more important than human extinction. Though it seems like the public is already moving on and correctly focusing on the real issues.

Re: Statement on AI Risk

#520

Post by pixl97 » Tue, May 30, 2023, 6:12 PM UTC

SanderNL wrote: post #362

sebzim4500 wrote: post #305

Earlier quoted context omitted.

To use Eliezer's analogy, this is like arguing about which move Stockfish would play to beat you in chess. If we're arguing about whether you can beat Stockfish, I will not be able to tell you the exact moves it will play but I am entirely justified in predicting that you will lose. Obviously we can imagine concrete ways a superintelligence might kill us all (engineer a virus, hack nuclear weapons, misinformation cam…

I understand but agentic/learning general intelligence has not been shown to exist, except ourselves. I’d say this is like worrying about deadly quantum laser weapons that will consume the planet when we are still in the AK47 phase. Edit: it could still be true though. I guess I like some more handholding and pre-chewing before giving governments and large corporations more ropes.

>except ourselves.

Good argument, lets ignore the (human) elephant in the room!

>worrying about deadly quantum laser weapons

If humans were shooting smaller less deadly quantum lasers out of their eyes I'd be very fucking worried that we'd make a much more powerful artificial version.

Tell me why do you think humans are the pinnacle of intelligence? What was the evolutionary requirement that somehow pushed us to this level?

You simply cannot answer that last question. Humans have a tiny power budget. We have a fit out of the birth canal limitation that cuts down on our brain size. We have a "don't starve to death" evolutionary pressure that was the biggest culling factor of all up to about 150 years ago. The idea that we couldn't build a better intelligence optimized system than nature is foreign to me, nature simply was not trying to achieve that goal.