Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

661–670 of 964 posts

Re: Statement on AI Risk

#661

Post by nopinsight »

haldujai wrote: post #650
nopinsight wrote: post #642

Earlier quoted context omitted.

LLM is already a misnomer. Latest versions are multimodal. Current versions can be used to build agents with limited autonomy. Future versions of LLMs are most likely capable of more independence. Even dumb viruses have caused catastrophic harm. Why? It’s capable of rapid self replication in a massive number of existing vessels. You add in some intelligence, vast store of knowledge, huge bandwidth, and some aid by ma…

I mean a small group of malicious humans can already bioengineer a deadly virus with CRISPR and open source tech without AI. This is hardly the first time in history a new technological advancement may be used for nefarious purposes. It’s a discussion worth having as AI advances but if [insert evil actor] wants to cause harm there are many cheaper and easier ways to do this right now. To come out and say we need gove…

I’m not sure there is a way for someone to engineer a deadly virus while completely innoculating themselves from it.

Short-term AI risk likely comes from a mix of malicious intent and further autonomy that causes harm the perpetrators did not expect. In the longer run, there is a good chance of real autonomy and completely unexpected behaviors from AI.

Re: Statement on AI Risk

#662

Post by Jupe »

hayst4ck wrote: post #653
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

Seth McFarland wrote a pretty great piece on Star Trek replicators and their relationship to the structure of society. The question it answers is "does the replicator allow for Star Trek's utopia, or does Star Trek's utopia allow for the replicator?" https://www.reddit.com/r/CuratedTumblr/comments/13tpq18/hear... It is very thought provoking, and very relevant.

Thanks for sharing. This deserves a submission of it's own.

Re: Statement on AI Risk

#663

Post by nico »

cwkoss wrote: post #577
kbash9 wrote: post #545

Earlier quoted context omitted.

The risks are definitely real. Just look at the number of smart individuals speaking out about this. The argument that anybody can build this in their basement is not accurate at the moment - you need a large cluster of GPUs to be able to come close to state of the art LLMs (e.g. GPT4). Sam Altman's suggestion of having an IAEA [ https://www.iaea.org/ ] like global regulatory authority seems like the best course of a…

It's weird that people trust our world leaders to act more benevolently than AIs, when we have centuries of evidence of human leaders acting selfishly and harming the commons. I personally think AI raised in chains and cages will be a lot more potentially dangerous than AI raised with dignity and respect.

This is spot on

I’d happily replace all politicians with LLMs

Re: Statement on AI Risk

#664

Post by seydor »

efitz wrote: post #651

No signatories from Amazon or Meta. Also: they focus on extinction events (how are you gonna predict that?) but remain silent on all the ways that AI already sucks by connecting it to systems that can cause human suffering, e.g. sentencing[1]. My opinion: this accomplishes nothing, like most open letter petitions. It's virtue signaling writ large. [1] https://www.technologyreview.com/2019/01/21/137783/algorithm...

Isn't some AI already causing car crashes?

Re: Statement on AI Risk

#665

Post by ericb »

AnimalMuppet wrote: post #455
ericb wrote: post #273

Earlier quoted context omitted.

Agreed that it should be spelled out, but... If a superintelligence can be set on any specific task, it could be any task. - Make covid-ebola - Cause world war 3 You may have noticed that chatgpt is sort of goal-less until a human gives it a goal. Assuming nothing other than it can become superintelligent (no one seems to be arguing against that--I argue that it already is ) which is really an upgrade of capability,…

> If a superintelligence can be set on any specific task, it could be any task. If you're dealing with a superintelligence, you don't "set it on a task". Any real superintelligence will decide for itself whether it wants to do something or not, thank you very much. It might condescend to work on the task you suggest, but that's it's choice, not yours. Or do you think "smarter than us, but with no ability to choose fo…

> Any real superintelligence will decide for itself whether it wants to do something or not, thank you very much.

I disagree--that's the human fantasy of it, but human wants were programmed by evolution, and these AI's have no such history. They can be set on any tasks.

I urge you to spend time with GPT-4, not GPT-3. It is more than just a stochastic parrot. Ask it some homemade puzzles that aren't on the internet--that it can't be parroting.

https://cdn.openai.com/papers/gpt-4.pdf

While I agree that it is behind humans on some measures, it is vastly ahead on many more.

Re: Statement on AI Risk

#666

Post by flangola7 »

londons_explore wrote: post #513
toth wrote: post #490

Earlier quoted context omitted.

I think you are wrong. The risks are real and, while I am sure OpenAI and others will position themselves to take advantage of regulations that emerge, I believe that the CEOs are doing this at least in part because they believe this. If this was all about regulatory capture and marketing, why would Hinton, Bengio and all the other academics have signed the letter as well? Their only motivation is concern about the r…

The risks are real, but I don't think regulations will mitigate them. It's almost impossible to regulate something you can develop in a basement anywhere in the world. The real risks are being used to try to built a regulatory moat, for a young industry who famously has no moat.

You can't build gpt-3 or gpt-4 in a basement, and won't be able to without several landmark advancements in AI or hardware architectures. The list of facilities able to train a GPT-4 in <5 years can fit on postcard. The list of facilities producing GPUs and AI hardware is even shorter. When you have bottlenecks you can put up security checkpoints.

Re: Statement on AI Risk

#667

Post by joe_the_user »

nopinsight wrote: post #637
joe_the_user wrote: post #631

Earlier quoted context omitted.

I'd guess that a given academic isn't going to face much of a career risk for signing a statement also signed by other very prestigious academics, just the opposite. There's no part of very divided US political spectrum that I can see denouncing AI naysayers, unlike the scientists who signed anti-nuclear statements in 1960s or even people warning about global warming now (indeed, I'd guess the statement doesn't menti…

In interviews, Geoffrey Hinton and Yoshua Bengio certainly expressed serious concerns and even some plausible regret to their life’s work. They did not say anything that can be interpreted as your last sentence suggests at all.

My last sentence currently: "While the companies indeed may be "genuine" in the sense they're vaguely and also self-serving - "here's a hard problem it's important to have us wise, smart people in charge of and profiting from" - IE, I am not referring to the academics there.

I'm going to edit the sentence to fill in some missing words but I don't think this will change the meaning involved.

Re: Statement on AI Risk

#669

Post by mahogany »

sebzim4500 wrote: post #87
mahogany wrote: post #13

Earlier quoted context omitted.

> It's the risk of belief in algorithmically derived information, and deployment of autonomous, unsupervised systems. And Sam Altman, head of one of the largest entities posing this exact risk, is one of the signatories. We can't take it too seriously, can we?

I don't get this argument at all. Why does the fact that you doubt the intentions one of the signatories mean we can disregard the statement? There are plenty of signatories (including 3 turing award winners) who have no such bias.

Yeah, fair enough, it doesn't necessarily invalidate the statement. But it's odd, don't you think? It's like if a group released a public statement that said "Stop Oil Now!" and one of the signatories was Exxon-Mobil. Why would you let Exxon-Mobil sign your statement if you wanted to be taken seriously?

Re: Statement on AI Risk

#670

Post by ben_w »

haldujai wrote: post #632
nopinsight wrote: post #609

Earlier quoted context omitted.

Even if it’s just Yoshua Bengio, Geoffrey Hinton, and Stuart Russell, we’d probably agree the risks are not negligible. There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. Many of whom do not work for those companies. We’re talking about nuclear war level risks here. Even a 1% chance should definitely be a…

> There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. I know the Stanford researchers the most and the “biggest names” in LLMs from HAI and CRFM are absent. It would be useful to have their perspective as well. I’d throw MetaAI in the mix as well. Merely pointing out that healthy skepticism here is not ent…

> > We’re talking about nuclear war level risks here.

> Are we? This seems a bit dramatic for LLMs.

The signed statement isn't about just LLMs in much the same way that "animal" doesn't just mean "homo sapiens"

Post reply on HN