Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

561–570 of 964 posts

Re: Statement on AI Risk

#561

Post by kalkin »

DonaldPShimoda wrote: post #471
haswell wrote: post #391

Earlier quoted context omitted.

> If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies Isn’t this to some degree exactly what all of these warnings about risk are leading to? And unlike nuclear weapons, there are massive monetary incentives that are directly at odds with behaving safely, and use cases that involve more than…

> It seems problematic to conclude there is no real risk purely on the basis of how software companies act. That is not the only basis. Another is the fact their lines of reasoning are literal fantasy. The signatories of this "statement" are steeped in histories of grossly misrepresenting and overstating the capabilities and details of modern AI platforms. They pretend to the masses that generative text tools like Ch…

Can you cite this history of "grossly misrepresenting" for some of the prominent academics on the list?

Honestly I'm a little skeptical that you could accurately attribute your scare-quoted "nearly sentient" to even Sam Altman. He's said a lot of things and I certainly haven't seen all of them, but I haven't seen him mix up intelligence and consciousness in that way.

Re: Statement on AI Risk

#562

Post by ppsreejith »

Interesting that nobody from Meta has signed this (tried searching for FAIR, Meta, Facebook) AND the fact that it seems to me that they're the ones releasing open code and model weights publicly (non commercial license though).

Also, judging by the comments here, perhaps people here would be less distrustful if the companies displayed more "skin in the game". For e.g: pledging to give up profiting from AI or committing all research to government labs (Maybe people can suggest better examples). Right now, it's not clear what the consequence of establishing the threat of AI as equivalent to nuclear war/pandemics would be. Would it later end up giving a powerful moat to these companies than they otherwise would have? Perhaps a lot of people are not comfortable with that outcome.

Re: Statement on AI Risk

#563

Post by bdlowery »

chriskanan wrote: post #505

I have mixed feelings about this. This letter is much better than the earlier one. There is a growing percentage of legitimate AI researchers who think that AGI could occur relatively soon (including me). The concern is that it could be given objectives intentionally or unintentionally that could lead to an extinction event. Certainly LLMs alone aren't anything close to AGIs, but I think that autoregressive training…

You’ve been watching too many movies.

Re: Statement on AI Risk

#564

Post by dauertewigkeit »

ppsreejith wrote: post #562

Interesting that nobody from Meta has signed this (tried searching for FAIR, Meta, Facebook) AND the fact that it seems to me that they're the ones releasing open code and model weights publicly (non commercial license though). Also, judging by the comments here, perhaps people here would be less distrustful if the companies displayed more "skin in the game". For e.g: pledging to give up profiting from AI or committi…

Yann LeCun has lots of influence at Meta and of the trio, he is the one who is completely dismissive of AGI existential risks.

Re: Statement on AI Risk

#565

Post by tdba »

The full statement so you don't have to click through:

> Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Re: Statement on AI Risk

#566

Post by [deleted] »

mitthrowaway2 wrote: post #441
juve1996 wrote: post #329

Earlier quoted context omitted.

What's more realistic: regulatory capture and US hegemony on AI or general intelligence destroying the world in the next 20 years? Go ahead and bet. I doubt you're putting your money on AGI.

Why would I put money on a bet that would only pay off when I, and everyone else, was dead?

[deleted]

Re: Statement on AI Risk

#567

Post by sangnoir »

kalkin wrote: post #211
b3nji wrote: post #77

Nonsense, the industry giants are just trying to scare the law makers to license the technology. Effectively, cutting out everyone else. Remember the Google note circulating saying "they have no moat", this is their moat. They have to protect their investment, we don't want people running this willy nilly for next to no cost on their own devices, God forbid!

This could be Google's motivation (although note that Google is not actually the market leader right now) but the risk could still be real. Most of the signatories are academics, for one thing, including two who won Turing awards for ML work and another who is the co-author of the standard AI textbook (at least when I was in school). You can be cynical about corporate motives and still worried. I personally am worrie…

> I personally am worried about AI partly because I am very cynical about how corporations will use it

This is the more realistic danger: I don't know if corporations are intentionally "controlling the narrative" by spewing unreasonable fears to distract from the actual dangers: AI + Capitalism + big tech/MNC + current tax regime = fewer white- & blue-collar jobs + increased concentration of wealth and a lower tax base for governments.

Having a few companies as AI gatekeepers will be terrible for society.

Re: Statement on AI Risk

#568

Post by pixl97 »

jononor wrote: post #477
rahmeero wrote: post #404

Earlier quoted context omitted.

There are many relevant things that already exist in the physical world and are not currently considered dangers: ecommerce, digital payments, doordash-style delivery, cross-border remittances, remote gig work, social media fanning extreme political views, event organizing. However, these are constituent elements that could be aggregated and weaponized by a maleficent AI.

Maleficent humans are constantly trying to use these elements for their own gain, often with little to no regards to other humans (especially out groups). This happens both individually, in small groups, in large organizations and even multiple organization colluding. Both criminal, terrorist, groups at war, along with legal organizations such as exploitative companies and regressive interest organizations, et.c.. An…

>Why and how are these mechanisms unsuitable for protecting against AI?

The rule of law prevented WWI and WWII, right? Oh, no it did not, tens to hundreds of millions died due to human stupidity and violence depending on what exactly you count in that age.

> Both criminal, terrorist, groups at war

Human organizations, especially criminal organizations have deep trust issues between agents in the organization. You never know if anyone else in the system is a defector. This reduces the openness and quantity of communication between agents. In addition you have agents that want to personally gain rather than benefit the organization itself. This is why Apple is a trillion dollar company following the law... mostly. Smart people can work together and 'mostly' trust the other person isn't going to screw them over.

Now imagine a superintelligent AI with a mental processing bandwidth of hundreds of the best employees at a company. Assuming it knows and trusts itself, then the idea of illegal activities being an internal risk disappears. You have something that operates more on the level of a hivemind toward a goal (what the limitations of hivemind versus selfish agents are is another very long discussion). What we ask here is if all the worlds best hackers got together, worked together unselfishly, and instigated an attack against every critical point they could find on the internet/real world systems at once, how much damage could they cause?

Oh, lets say you find the server systems the super intelligence is on, but the controller shuts it off and all the data has some kind of homomorphic encryption so that's useless to you. It's dead right? Na, they just load up the backup copy they have a few months later and it's party time all over again. Humans tend to remain dead after dying, AI? Well that is yet to bee seen.

Re: Statement on AI Risk

#569

Post by ever1337 »

boringuser2 wrote: post #393
falsaberN1 wrote: post #220

Earlier quoted context omitted.

And I don't get the opposed mindset, that AI is suddenly going to "become a real boy, and murder us all". Isn't it a funny coincidence how the popular opinion of AIs aligns perfectly with blockbusters and popular media ONLY? People are specifically wanting to prevent Skynet. The kicker (and irony to a degree) is that I really want sapient AI to exist. People being so influenced by fiction is something I see as a mena…

Very Dunning-Kruger post right here.

Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.

Re: Statement on AI Risk

#570

Post by nopinsight »

chefandy wrote: post #487
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

It’s worth noting also that many academics who signed the statement may face adverse issues like reputational risk as well as funding cut to their research programs if AI safety becomes an official policy.

For a large number of them, these risks are worth far more than any possible gain from signing it.

When a large number of smart, reputable people, including many with expert knowledge and little or negative incentives to act dishonestly, put their names down like this, one should pay attention.

Added:

Paul Christiano, a brilliant theoretical CS researcher who switched to AI Alignment several years ago, put the risks of “doom” for humanity at 46%.

https://www.lesswrong.com/posts/xWMqsvHapP3nwdSW8/my-views-o...

Post reply on HN