Statement on AI Risk • Hacker News Board

Re: Statement on AI Risk

#391

Post by haswell » Tue, May 30, 2023, 4:58 PM UTC

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

> If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies

Isn’t this to some degree exactly what all of these warnings about risk are leading to?

And unlike nuclear weapons, there are massive monetary incentives that are directly at odds with behaving safely, and use cases that involve more than ending life on earth.

It seems problematic to conclude there is no real risk purely on the basis of how software companies act.

Re: Statement on AI Risk

#392

Post by Symmetry » Tue, May 30, 2023, 4:58 PM UTC

skepticATX wrote: post #72

apsec112 wrote: post #51

A lot of the responses to this seem like Bulverism, ie., trying to refute an argument by psychoanalyzing the people who argue it: "Suppose I think, after doing my accounts, that I have a large balance at the bank. And suppose you want to find out whether this belief of mine is "wishful thinking." You can never come to any conclusion by examining my psychological condition. Your only chance of finding out is to sit do…

But what argument is there to refute? It feels like Aquinas “proving” God’s existence by stating that it is self evident. They can’t point to an existing system that poses existential risk, because it doesn’t exist. They can’t point to a clear architecture for such a system, because we don’t know how to build it. So again, what can be refuted?

The idea is that if you build a system that poses an existential risk you want to be reasonably sure it's safe before you turn it on, not afterwards. It would have been irresponsible for the scientists at Los Alamost to do the math on whether an atomic explosion would create a sustained fusion reaction in the atmosphere until after their first test, for example.

I don't think it's possible for a large language model, operating in a conventional feed forward way, to really pose a significant danger. But I do think it's hard to say exactly what advances could lead to a dangerous intelligence and with the current state of the art it looks to me at least like we might very well be only one breakthrough away from that. Hence the calls for prudence.

The scientists creating the atomic bomb knew a lot more about what they were doing than we do. Their computations sometimes gave the wrong result, see Castle Bravo, but had a good framework for understanding everything that was happening. We're more like cavemen who've learned to reliably make fire but still don't understand it. Why can current versions of GPT reliably add large numbers together when previous versions couldn't? We're still a very long way away from being able to answer questions like that.

Re: Statement on AI Risk

#393

Post by boringuser2 » Tue, May 30, 2023, 4:59 PM UTC

falsaberN1 wrote: post #220

patrec wrote: post #58

Earlier quoted context omitted.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

And I don't get the opposed mindset, that AI is suddenly going to "become a real boy, and murder us all". Isn't it a funny coincidence how the popular opinion of AIs aligns perfectly with blockbusters and popular media ONLY? People are specifically wanting to prevent Skynet. The kicker (and irony to a degree) is that I really want sapient AI to exist. People being so influenced by fiction is something I see as a mena…

Very Dunning-Kruger post right here.

Re: Statement on AI Risk

#394

Post by holmesworcester » Tue, May 30, 2023, 4:59 PM UTC

escape-big-tech wrote: post #377

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

Agreed. If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement

> If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement

This statement contains a bunch of hidden assumptions:

1. That they believe their stopping will address the problem. 2. That they believe the only choice is whether or not to stop. 3. That they don't think it's possible to make AI safe through sufficient regulation. 4. That they don't see benefits to pursuing AI that could outweigh risks.

If they believe any of these things, then they could believe the risks were real and also not believe that stopping was the right answer.

And it doesn't depend on whether any of these beliefs are true: it's sufficient for them to simply believe one of them and the assumptions your statement depends on break down.

Re: Statement on AI Risk

#395

Post by mark_l_watson » Tue, May 30, 2023, 5:00 PM UTC

We all have different things we worry about. My family and old friends have heard me talking about AI for 40 years. When asked about dangers of AI, I only talk about humans using AI to fake interactions with people at scale without divulging the identity as an AI, fake political videos, and individualized ‘programming’ of the public by feeding them personal propaganda and sales pitches.

I never talk about, or worry about, the ‘killer robot’ or AIs taking over infrastructure scenarios. I hope I am not wrong about these types of dangers.

Re: Statement on AI Risk

#396

Post by boringuser2 » Tue, May 30, 2023, 5:00 PM UTC

shafyy wrote: post #32

The issue I take with these kind of "AI safety" organizations is that they focus on the wrong aspects of AI safety. Specifically, they run this narrative that AI will make us humans go extinct. This is not a real risk today. Real risks are more in the category of systemic racism and sexism, deep fakes, over reliance on AI etc. But of course, "AI will humans extinct" is much sexier and collects clicks. Therefore, the…

What is "systemic racism"? How is it a risk?

Don't bother explaining, we already know it's unfalfisiable.

Re: Statement on AI Risk

#397

Post by lumost » Tue, May 30, 2023, 5:01 PM UTC

Who wants to build a black market AI?

Evidence points that this technology is going to become cheap, fast. There is an existential risk to the very notion of search as a business model, within the next ~5 years we are almost certain to have an app which is under 20 GB in size and has an effective index of the useful/knowledgable portion of the internet and is able to run on most laptops/phones.

At best, regulating this will be like trying to regulate torrents in the 2000s, building a bespoke/crappy AI will be the new "learn HTML" for high school kids.

Re: Statement on AI Risk

#398

Post by lxnn » Tue, May 30, 2023, 5:01 PM UTC

endisneigh wrote: post #20

> Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. risk of extinction due to AI? people have been reading too much science fiction. I would love to hear a plausible story of how AI will lead to human extinction that wouldn't happen with traditional non-AI tech. for the sake of conversation let's say non-AI tech is any broadly…

The emergence of something significantly more intelligent than us whose goal are not perfectly aligned with ours poses a pretty clear existential risk. See, for example, the thousands of species made extinct by humans.

Re: Statement on AI Risk

#399

Post by MrScruff » Tue, May 30, 2023, 5:01 PM UTC

JamesLeonis wrote: post #308

MrScruff wrote: post #188

I find it quite extraordinary how many on here are dismissing that there is any risk at all. I also find statements like Yann LeCunn's that "The most common reaction by AI researchers to these prophecies of doom is face palming." to be lacking in awareness. "Experts disagree on risk of extinction" isn't quite as reassuring as he thinks it is. The reality is, despite the opinions of the armchair quarterbacks commentin…

> I find it quite extraordinary how many on here are dismissing that there is any risk at all. The fear over AI is a displaced fear of unaccountable social structures with extinction-power that currently exist and we allow to continually exist . Without these structures AI is harmless to the species, even superintelligence. Your (reasonable) counter-argument might be that somebody (like, say, my dumb self) accidental…

I don't really have a notion of whether an actual AGI would have a desire to kill all humans. I do however think that one entity seeking to create another entity that it can control, yet is more intelligent than it, seems arbitrarily challenging in the long run.

I think having a moratorium on AI development will be impossible to enforce, and as you stretch the timeline out, these negative outcomes become increasingly likely as the technical barriers to entry continue to fall.

I've personally assumed this for thirty years, the only difference now is that the timeline seems to be accelerating.

Re: Statement on AI Risk

#400

Post by Planktonne » Tue, May 30, 2023, 5:02 PM UTC

qt31415926 wrote: post #354

Planktonne wrote: post #315

Earlier quoted context omitted.

LLMs are in her field; they are one of her research topics and they're definitely getting funding. We absolutely should not be ignoring research that doesn't support popular narratives; dismissing her work because it is critical of LLMs is not reasonable.

In her field doesn't mean that's what she researches, LLMs are loosely in her field but the methods are completely different. Computational linguistics != deep learning. Deep learning does not directly use concepts from linguistics, semantics, grammars or grammar engineering, which is what Emily was researcing for the past decades. It's the same thing as saying a number theorist and a set theorist are in the same fie…

They are what she researches though. She has published research on them.

LLMs don't directly use concepts from linguistics but they do produce and model language/grammar; it's entirely valid to use techniques from those fields to evaluate them, which is what she does. In the same vein, though a self-driving car doesn't work the same way as a human driver does, we can measure their performance on similar tasks.