Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

631–640 of 964 posts

Re: Statement on AI Risk

#631

Post by joe_the_user »

nopinsight wrote: post #570
chefandy wrote: post #487

Earlier quoted context omitted.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

It’s worth noting also that many academics who signed the statement may face adverse issues like reputational risk as well as funding cut to their research programs if AI safety becomes an official policy. For a large number of them, these risks are worth far more than any possible gain from signing it. When a large number of smart, reputable people, including many with expert knowledge and little or negative incenti…

I'd guess that a given academic isn't going to face much of a career risk for signing a statement also signed by other very prestigious academics, just the opposite. There's no part of very divided US political spectrum that I can see denouncing AI naysayers, unlike the scientists who signed anti-nuclear statements in 1960s or even people warning about global warming now (indeed, I'd guess the statement doesn't mention climate change 'cause it's still a sore point).

Moreover, talking about existential risk involves the assumption the current tech is going to continue to affect more and more fields rather than peaking at some point - this assumption guarantees more funding along with funding for risk.

All that said, I don't necessarily think the scientists involved are insincere. Rather, I would expect they're worried and signed this vague statement because it was something that might get traction. While the companies indeed may be "genuine" in the sense they're vaguely [concerned - edit] and also self-serving - "here's a hard problem it's important to have us wise, smart people in charge of and profiting from"

Re: Statement on AI Risk

#632

Post by haldujai »

nopinsight wrote: post #609
haldujai wrote: post #595

Earlier quoted context omitted.

Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left. Notably missing representation from the Stanford NLP (edit: I missed that Diyi Yang is a signatory on first read) and NYU groups who’s perspective I’d also be interested in hearing. Not committing one way or another regarding the intent with this but it’s not as diverse an aca…

Even if it’s just Yoshua Bengio, Geoffrey Hinton, and Stuart Russell, we’d probably agree the risks are not negligible. There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. Many of whom do not work for those companies. We’re talking about nuclear war level risks here. Even a 1% chance should definitely be a…

> There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well.

I know the Stanford researchers the most and the “biggest names” in LLMs from HAI and CRFM are absent. It would be useful to have their perspective as well.

I’d throw MetaAI in the mix as well.

Merely pointing out that healthy skepticism here is not entirely unwarranted.

> We’re talking about nuclear war level risks here.

Are we? This seems a bit dramatic for LLMs.

Re: Statement on AI Risk

#633

Post by tech_ken »

dwaltrip wrote: post #198
sf4lifer wrote: post #139

LLMs are just text prediction. I don't see the linear path from LLM to AGI. Was there similar hysteria when the calculator or PC first came out?

It’s not just random text, they are predicting writings and documents produced by humans. They are “language” models. Language is used to say things about the world. This means that predicting language extremely well is best done through acquiring an understanding of the world. Take a rigorous, well-written textbook. Predicting a textbook is like writing a textbook. To write a good textbook, you need to be an expert…

>Predicting a textbook is like writing a textbook

HN AGI discourse is full of statements like this (eg. all the stuff about stochastic parrots), but to me this seems massively non-obvious. Mimicking and rephrasing pre-written text is very different from conceiving of and organizing information in new ways. Textbook authors are not simply transcribing their grad school notes down into a book and selling it. They are surveying a field, prioritizing its knowledge content based on an intended audience, organizing said information based on their own experience with and opinions on the learning process, and presenting the knowledge in a way which engages the audience. LLMs are a long way off from this latter behavior, as far as I can tell.

> The best language models (eg GPT-4) have some understanding of the world

This is another statement that I see variants of a lot, but which seems to way overstate the case. IMO it's like saying that a linear regression "understands" econometrics or a series of coupled ODEs "understands" epidemiology; it's at best an abuse of terminology and at worst a complete misapplication of the term. If I take a picture of a page of a textbook the resulting JPEG is "reproducing" the text, but it doesn't understand the content it's presenting to me in a meaningful way. Sure it has primitives with which it can store the content, but human understanding covers a far richer set of behaviors than merely storing/compressing training inputs. It implies being able to generalize and extrapolate the digested information in novel situations, effectively growing one's own training data. I don't see that behavior in GPT-4

Re: Statement on AI Risk

#634

Post by scrum-treats »

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

> This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power.

This really is the crux of the issue isn't it? All this pushback for the first petition, because "Elon Musk," but now GPT wonder Sam Altman "testifies" that he has "no monetary interest in OpenAI" and quickly follows up his proclamation with a second "Statement on AI Risks." Oh, and let's not forget, "buy my crypto-coin"!

But Elon Musk... Ehh.... Looking like LOTR out here with "my precious" AGI on the brain.

Not to downplay the very serious risk at all. Simply echoing the sentiment that we would do well to stay objective and skeptical of ALL these AI leaders pushing new AI doctrine. At this stage, it's a policy push and power grab.

Re: Statement on AI Risk

#635

Post by slg »

revelio wrote: post #584
kbash9 wrote: post #545

Earlier quoted context omitted.

The risks are definitely real. Just look at the number of smart individuals speaking out about this. The argument that anybody can build this in their basement is not accurate at the moment - you need a large cluster of GPUs to be able to come close to state of the art LLMs (e.g. GPT4). Sam Altman's suggestion of having an IAEA [ https://www.iaea.org/ ] like global regulatory authority seems like the best course of a…

> The risks are definitely real. Just look at the number of smart individuals speaking out about this. In our society smart people are strongly incentivized to invent bizarre risks in order to reap fame and glory. There is no social penalty if those risks never materialize, turn out to be exaggerated or based on fundamental misunderstanding. They just shrug and say, well, better safe than sorry, and everyone lets the…

>In our society smart people are strongly incentivized to invent bizarre risks in order to reap fame and glory. There is no social penalty if those risks never materialize, turn out to be exaggerated or based on fundamental misunderstanding.

Are people here not old enough to remember how much Ralph Nader and Al Gore were mocked for their warnings despite generally being right?

Re: Statement on AI Risk

#636

Post by worik »

Where were these people when their algorithms were:

• leading people down YouTube rabbit holes? • amplifying prejudice in the legal system? • wrecking teenagers lives on social media?

The list goes on

They were nowhere, or were getting stinking rich.

Hypocrites

Re: Statement on AI Risk

#637

Post by nopinsight »

joe_the_user wrote: post #631
nopinsight wrote: post #570

Earlier quoted context omitted.

It’s worth noting also that many academics who signed the statement may face adverse issues like reputational risk as well as funding cut to their research programs if AI safety becomes an official policy. For a large number of them, these risks are worth far more than any possible gain from signing it. When a large number of smart, reputable people, including many with expert knowledge and little or negative incenti…

I'd guess that a given academic isn't going to face much of a career risk for signing a statement also signed by other very prestigious academics, just the opposite. There's no part of very divided US political spectrum that I can see denouncing AI naysayers, unlike the scientists who signed anti-nuclear statements in 1960s or even people warning about global warming now (indeed, I'd guess the statement doesn't menti…

In interviews, Geoffrey Hinton and Yoshua Bengio certainly expressed serious concerns and even some plausible regret to their life’s work. They did not say anything that can be interpreted as your last sentence suggests at all.

Re: Statement on AI Risk

#638

Post by ericb »

agnosticmantis wrote: post #574
ericb wrote: post #273

Earlier quoted context omitted.

Agreed that it should be spelled out, but... If a superintelligence can be set on any specific task, it could be any task. - Make covid-ebola - Cause world war 3 You may have noticed that chatgpt is sort of goal-less until a human gives it a goal. Assuming nothing other than it can become superintelligent (no one seems to be arguing against that--I argue that it already is ) which is really an upgrade of capability,…

Respectfully, just because we can put together some words doesn’t mean they make a meaningful expression, even if everybody keeps repeating them as if they did make sense: e.g. an omnipotent God, artificial general intelligence, super-intelligence, infinitely many angels sitting on the tip of a needle, etc.

Is your comment self-referential?

Re: Statement on AI Risk

#639

Post by hackermatic »

haswell wrote: post #418
goolulusaurs wrote: post #388

Throughout history there have been hundreds, if not thousands of examples of people and groups who thought the end of the world was imminent. So far, 100% of those people have been wrong. The prior should be that the people who believe in AI doomsday scenarios are wrong also, unless and until there is very strong evidence to the contrary. Vague theoretical arguments are not sufficient, as there are many organizations…

If you were to apply this argument to the development of weapons, it’s clear that there is a threshold that is eventually reached that fundamentally alters the stakes. A point past which all prior assumptions about risk no longer apply. It also seems very problematic to conclude anything meaningful about AI when realizing that a significant number of those examples are doomsday cults, the very definition of extremist…

Although I think the existential risk of AI isn't a priority yet, this reminds me of a quote I heard for the first time yesterday night, from a draft script for 2001: A Space Odyssey[0]:

> There had been no deliberate or accidental use of nuclear weapons since World War II and some people felt secure in this knowledge. But to others, the situation seemed comparable to an airline with a perfect safety record; it showed admirable care and skill but no one expected it to last forever.

[0] https://movies.stackexchange.com/a/119598

Re: Statement on AI Risk

#640

Post by mcguire »

Is it ironic that they start with, "Even so, it can be difficult to voice concerns about some of advanced AI’s most severe risks," and then write about "the risk of extinction from AI" which is,

a) the only risk of AI that seems to get a lot of public discussion, and

b) completely ignores the other, much more likely risks of AI.

Post reply on HN