Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

481–490 of 964 posts

Re: Statement on AI Risk

#481

Post by ethanbond »

SanderNL wrote: post #362
sebzim4500 wrote: post #305

Earlier quoted context omitted.

To use Eliezer's analogy, this is like arguing about which move Stockfish would play to beat you in chess. If we're arguing about whether you can beat Stockfish, I will not be able to tell you the exact moves it will play but I am entirely justified in predicting that you will lose. Obviously we can imagine concrete ways a superintelligence might kill us all (engineer a virus, hack nuclear weapons, misinformation cam…

I understand but agentic/learning general intelligence has not been shown to exist, except ourselves. I’d say this is like worrying about deadly quantum laser weapons that will consume the planet when we are still in the AK47 phase. Edit: it could still be true though. I guess I like some more handholding and pre-chewing before giving governments and large corporations more ropes.

Or it’s like worrying about an arms race toward civilization-ending arsenals after seeing the Trinity test… which… was the correct response.

We don’t know it’s possible to build superintelligences but so far we don’t have a good reason to think we can’t and we have complete certainty that humans will spend immense, immense resources getting as close as they can as fast as they can.

Very different from the lasers.

Re: Statement on AI Risk

#482

Post by AlexandrB »

holmesworcester wrote: post #378
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

As others have pointed out, there are many on this list (Bruce Schneier, for example) who do not stand to benefit from AI marketing or regulatory capture. Anyone upvoting this comment should take a long look at the names on this letter and realize that many are not conflicted. Many signers of this letter are more politically sophisticated than the average HN commenter, also. So sure, maybe they're getting rolled by m…

I definitely agree that names like Hinton, Schneier, and Norvig add a lot of weight here. The involvement of OpenAI muddies the water a lot though and it's not at all clear what is meant by "risk of extinction". It sounds scary, but what's the mechanism? The safe.ai website lists 8 risks, but these are quite vague as well, with many alluding to disruption of social order as the primary harm. If safe.ai knows something we don't, I wish they could communicate it more clearly.

I also find it somewhat telling that something like "massive wealth disparity" or "massive unemployment" are not on the list, when this is a surefire way to create a highly unstable society and a far more immediate risk than AI going rogue. Risk #5 (below) sort of alludes to it, but misses the mark by pointing towards a hypothetical "regime" instead of companies like OpenAI.

> Value Lock-In

> Highly competent systems could give small groups of people a tremendous amount of power, leading to a lock-in of oppressive systems.

> AI imbued with particular values may determine the values that are propagated into the future. Some argue that the exponentially increasing compute and data barriers to entry make AI a centralizing force. As time progresses, the most powerful AI systems may be designed by and available to fewer and fewer stakeholders. This may enable, for instance, regimes to enforce narrow values through pervasive surveillance and oppressive censorship. Overcoming such a regime could be unlikely, especially if we come to depend on it. Even if creators of these systems know their systems are self-serving or harmful to others, they may have incentives to reinforce their power and avoid distributing control.

Re: Statement on AI Risk

#483

Post by pixl97 »

falsaberN1 wrote: post #220
patrec wrote: post #58

Earlier quoted context omitted.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

And I don't get the opposed mindset, that AI is suddenly going to "become a real boy, and murder us all". Isn't it a funny coincidence how the popular opinion of AIs aligns perfectly with blockbusters and popular media ONLY? People are specifically wanting to prevent Skynet. The kicker (and irony to a degree) is that I really want sapient AI to exist. People being so influenced by fiction is something I see as a mena…

A counter point here is you're ignoring all the boring we all die scenarios that are completely possible but too boring to make a movie about.

The AI hooked to a gene sequencer/printer test lab is something that is nearly if not completely possible now. It's something that can be relatively small in size compared with the facilities needed to make most weapons of mass destruction. It's something that is highly iterative, and parallelizable. And it's something powerful enough that if targeting at the correct things (kill all rice, kill all X people) that it easily spills over in to global conflict.

Re: Statement on AI Risk

#484

Post by yellow_postit »

computerphage wrote: post #387
escape-big-tech wrote: post #377

Earlier quoted context omitted.

Agreed. If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement

Geoffrey Hinton quit google.

It’s hard not to look at his departure through a cynical lens. He’s not been supportive of other critics, both from and outside of Google. He also wants to use his history to (rightfully) claim expertise and power but not to offer solutions.

Re: Statement on AI Risk

#485

Post by logicchains »

ChatGTP wrote: post #71
vegabook wrote: post #4

[Un]intended consequence: "AI is too dangerous for you little guys. Leave it to we the FAANGs" - rubber stamped with legislation.

What would Max Tegmark, Geoffrey Hinton, Yohshua Bengio (to name a few) have absolutely anything to do with FAANG ? They're completely independent AI researchers and geniuses spending their own free time on trying to warn you and others of the dangers of the technology they've created to help keep the world safer. Seems like you're taking a far too overly cynical position ?

[flagged]

Re: Statement on AI Risk

#486

Post by NumberWangMan »

dopamean wrote: post #431
escape-big-tech wrote: post #377

Earlier quoted context omitted.

Agreed. If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement

I agree that nothing about the statement makes me think the risks are real however I disagree that if the risks are real these companies would stop working on their product. I think more realistically they'd shut up about the risk and downplay it a lot. Much like the oil industry did wrt climate change going back to the 70's.

Oil industries downplaying the risks makes a lot more sense. If you think that climate change will happen, but it'll happen after you're dead, and you'll be able to leave your kids a big inheritance so they'll be able to buy their way out of the worst of it, and eventually the government will get the message and stop us all using fossil fuels anyway, then you try to profit as much as you can in the short term.

With AGI existential risk, its likely to happen on a much shorter timescale, and it seems likely you won't be able to buy your way out of it.

Re: Statement on AI Risk

#487

Post by chefandy »

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't sign their name to any ol' open letter, especially if including their organizational affiliations. If this is a marketing ploy, that must have been a monster because even if they were walking around handing out checks for 25k I doubt they'd have gotten a good chunk of these folks.

Re: Statement on AI Risk

#488

Post by duvenaud »

enord wrote: post #465
duvenaud wrote: post #351

Earlier quoted context omitted.

This is a great point. But I'd say that capable entities have a habit of turning themselves into agents. A great example is totalitarian governments. Even if every single citizen hates the regime, they're still forced to support it. You could similarly ask: Why would we ever build a government or institution that cared more about its own self-preservation than its original mission? The answer is: Natural selection fa…

Now agency is an end unto itself I wholeheartedly agree. I feel though, that any worry about the agency of supercapable computer systems is premature until we see even the tiniest— and I mean really anything at all— sign of their agency. Heck, even agency _in theory_ would suffice, and yet: nada.

I'm confused. You agree that we're surround by naturally-arising, self-organizing agents, both biological and institutional. People are constantly experimenting with agentic AIs of all kinds. There are tons of theoretical characterizations of agency and how it's a stable equilibrium. I'm not sure what you're hoping for if none of these are reasons to even worry.

Re: Statement on AI Risk

#489

Post by NumberWangMan »

munificent wrote: post #379
aero-deck wrote: post #154

Earlier quoted context omitted.

It doesn't matter who wrote it, it got picked up, had a good argument and affected market opinion. The execs now need to respond to it. Humans also don't grasp that things can improve exponentially until they stop improving exponentially. This belief that AGI is just over the hill is sugar-water for extracting more hours from developers. The nuclear bomb was also supposed to change everything. But in the end nothing…

> The nuclear bomb was also supposed to change everything. But in the end nothing changed, we just got more of the same. It is hard for me to imagine a statement more out of touch with history than this. All geopolitical history from WWII forward is profoundly affected by the development of the bomb. I don't even know where to begin to argue against this. Off the top of my head: 1. What would have happened between Ja…

Not to mention how close the USA and Soviet Union were to a nuclear exchange: https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...

Re: Statement on AI Risk

#490

Post by toth »

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I think you are wrong. The risks are real and, while I am sure OpenAI and others will position themselves to take advantage of regulations that emerge, I believe that the CEOs are doing this at least in part because they believe this.

If this was all about regulatory capture and marketing, why would Hinton, Bengio and all the other academics have signed the letter as well? Their only motivation is concern about the risks.

Worry about AI x-risk is slowly coming into the Overton window, but until very recently you could get ridiculed by saying publicly you took it seriously. Academics knew this and still came forward - all the people who think its nonsense should at least try to consider they are earnest and could be right.

Post reply on HN