Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

681–690 of 964 posts

Re: Statement on AI Risk

#681

Post by skybrian »

nopinsight wrote: post #654
chefandy wrote: post #487

Earlier quoted context omitted.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

Here’s why AI risks are real, even if our most advanced AI is merely a ‘language’ model: Language can represent thoughts and some world models. There is strong evidence that LLMs contain some representation of world models it learned from text. Moreover, LLM is already a misnomer; latest versions are multimodal. Current versions can be used to build agents with limited autonomy. Future versions of LLMs are most likel…

A lot of things are called "world models" that I would consider just "models" so it depends on what you mean by that. But what do you consider to be strong evidence? The Othello paper isn't what I'd call strong evidence.

Re: Statement on AI Risk

#682

Post by patrec »

patch_cable wrote: post #375
patrec wrote: post #58

Earlier quoted context omitted.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

> How can it not be obvious to you It isn't obvious to me. And I've yet to read something that spills out the obvious reasoning. I feel like everything I've read just spells out some contrived scenario, and then when folks push back explaining all the reasons that particular scenario wouldn't come to pass, the counter argument is just "but that's just one example!" without offering anything more convincing. Do you ha…

OK, which of the following propositions do you disagree with?

1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas.

2. The fact that AIs have some inherent scalability, speed, cost, reliability and compliance advantages over humans means that many undesirable things that could previously not be done at all or at least not done at scale are becoming both feasible and cost-effective. Examples would include 24/7 surveillance with social desirability scoring based on a precise ideological and psychological profile derived from a comprehensive record of interactions, fine-tuned mass manipulation and large scale plausible falsification of the historical record. Given the general rise of authoritarianism, this is pretty worrying.

3. On the other hand the rapid progress and enormous investment we've been seeing makes it very plausible that before too long we will, in fact, see AIs that outperform humans on most tasks.

4. AIs that are much smarter than any human pose even graver dangers.

5. Even if there is a general agreement that AIs pose grave or even existential risks, states, organizations and individuals will are all incentivized to still seek to improve their own AI capabilities, as doing so provides an enormous competitive advantage.

6. There is a danger of a rapid self-improvement feedback loop. Humans can reproduce, learn new and significantly improve existing skills, as well as pass skills on to others via teaching. But there are fundamental limits on speed and scale for all of these, whereas it's not obvious at all how an AI that has reached super-human level intelligence would be fundamentally prevented from rapidly improving itself further, or produce millions of "offspring" that can collaborate and skill-exchange extremely efficiently. Furthermore, since AIs can operate at completely different time scales than humans, this all could happen extremely rapidly, and such a system might very quickly become much more powerful than humanity and the rest of AIs combined.

I think you only have to subscribe a small subset of these (say 1.&2.) to conclude that "AI is an uniquely powerful and thus uniquely dangerous technology" obviously follows.

For the stronger claim of existential risk, have you read the lesswrong link posted elsewhere in this discussion?

https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a... ?

Re: Statement on AI Risk

#683

Post by [deleted] »

nopinsight wrote: post #609
haldujai wrote: post #595

Earlier quoted context omitted.

Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left. Notably missing representation from the Stanford NLP (edit: I missed that Diyi Yang is a signatory on first read) and NYU groups who’s perspective I’d also be interested in hearing. Not committing one way or another regarding the intent with this but it’s not as diverse an aca…

Even if it’s just Yoshua Bengio, Geoffrey Hinton, and Stuart Russell, we’d probably agree the risks are not negligible. There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. Many of whom do not work for those companies. We’re talking about nuclear war level risks here. Even a 1% chance should definitely be a…

[deleted]

Re: Statement on AI Risk

#684

Post by haldujai »

ben_w wrote: post #670
haldujai wrote: post #632

Earlier quoted context omitted.

> There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. I know the Stanford researchers the most and the “biggest names” in LLMs from HAI and CRFM are absent. It would be useful to have their perspective as well. I’d throw MetaAI in the mix as well. Merely pointing out that healthy skepticism here is not ent…

> > We’re talking about nuclear war level risks here. > Are we? This seems a bit dramatic for LLMs. The signed statement isn't about just LLMs in much the same way that "animal" doesn't just mean "homo sapiens"

I used LLM because the people shouting the loudest come from a LLM company which claimed their newest language model can be used to create bioweapons in their whitepaper.

Semantics aside the recent interest in AI risk was clearly stimulated by LLMs and the camp that believes this is the path to AGI which may or may not be true depending who you ask.

Re: Statement on AI Risk

#685

Post by jumelles »

pixl97 wrote: post #483
falsaberN1 wrote: post #220

Earlier quoted context omitted.

And I don't get the opposed mindset, that AI is suddenly going to "become a real boy, and murder us all". Isn't it a funny coincidence how the popular opinion of AIs aligns perfectly with blockbusters and popular media ONLY? People are specifically wanting to prevent Skynet. The kicker (and irony to a degree) is that I really want sapient AI to exist. People being so influenced by fiction is something I see as a mena…

A counter point here is you're ignoring all the boring we all die scenarios that are completely possible but too boring to make a movie about. The AI hooked to a gene sequencer/printer test lab is something that is nearly if not completely possible now. It's something that can be relatively small in size compared with the facilities needed to make most weapons of mass destruction. It's something that is highly iterat…

Okay, so AI has access to a gene printer. Then what?

Re: Statement on AI Risk

#686

Post by agnosticmantis »

ericb wrote: post #638
agnosticmantis wrote: post #574

Earlier quoted context omitted.

Respectfully, just because we can put together some words doesn’t mean they make a meaningful expression, even if everybody keeps repeating them as if they did make sense: e.g. an omnipotent God, artificial general intelligence, super-intelligence, infinitely many angels sitting on the tip of a needle, etc.

Is your comment self-referential?

I don’t think so. If you look at the thread, it’s already devolved into an analogue of “what happens when an irresistible force meets an immovable obstacle?”

(Specifically I mean the comment about another “super-intelligence” preventing whatever your flavor of “super-intelligence” does.)

At this point we can safely assume words have lost their connection to physical reality. No offense to you, just my two-cent meta comment.

Re: Statement on AI Risk

#687

Post by [deleted] »

ericb wrote: post #646
lucisferre wrote: post #374

Earlier quoted context omitted.

Are you really arguing ChatGPT is already super-intelligent? What is your basis for this conclusion? And many people argue against the idea that GPT is already super intelligent or even can become so at this stage of development and understanding. In fact as far as I can tell it is the consensus right now of experts and it's creators. https://www.calcalistech.com/ctechnews/article/nt9qoqmzz

If super means "surpassing normal human intelligence" then, YES. Take a look at the table in this article. If a human did that, was fluent in every language and coded in every language, we'd say they were superhuman, no? https://cdn.openai.com/papers/gpt-4.pdf

[deleted]

Re: Statement on AI Risk

#688

Post by revelio »

pixl97 wrote: post #659
revelio wrote: post #608

Earlier quoted context omitted.

AI also has a power budget. It has to fit in a datacenter. Inconveniently for AI, that power budget is controlled by us.

Do you remember the days the computing power that's in your pocket took up an entire floor of a building? Because I do. If that is somehow the only barrier between humanity and annihilation, then things don't bode well for us.

Sure. Despite all that progress, computers still have an off switch and power efficiency still matters. It actually matters more now than in the past.

Re: Statement on AI Risk

#689

Post by haswell »

goolulusaurs wrote: post #539
staunton wrote: post #517

Earlier quoted context omitted.

That argument is still invalid because in scenario 2 we would not be having this discussion. No conclusions can be drawn from such past discourse about the likelihood of definite and complete extinction. Not that, I hope, anyone expected a strong argument to be had there. It seems reasonably certain to me that humanity will go extinct one way or another eventually. That is also not a good argument in this situation.

It depends on what you mean by "this discussion", but I don't think that follows. If for example, we were in scenario 2 and it was still the case that a large number of people thought AI doomsday was a serious risk, then that would be a much stronger argument for taking the idea of AI doomsday seriously. If on the other hand we are in scenario 1, where there is a long history of people falling prey to apocalypticism,…

> If on the other hand we are in scenario 1, where there is a long history of people falling prey to apocalypticism, then that means any new doomsday claims are also more likely to be a result of apocalypticism.

If you're blindly evaluating the likelihood of any random claim without context, sure.

But like the boy who cried wolf, there is a potential scenario where the likelihood that it's not true has no bearing on what actually happens.

Arguably, claims about doomsday made now by highly educated people are more interesting than claims made 100/1000/10000 years ago. Over time, the growing collective knowledge of humanity increases and with it, the plausibility of those claims because of our increasing ability to accurately predict outcomes based on our models of the world.

e.g. after the introduction of nuclear weapons, a claim about the potentially apocalyptic impact of war is far more plausible than it would have been prior.

Similarly, we can now estimate the risk of passing comets/asteroids, and if we identify one that's on a collision course, we know that our technology makes it worth taking that risk more seriously than someone making a prediction in an era before we could possible know such things.

Re: Statement on AI Risk

#690

Post by PoignardAzur »

JamesLeonis wrote: post #308
MrScruff wrote: post #188

I find it quite extraordinary how many on here are dismissing that there is any risk at all. I also find statements like Yann LeCunn's that "The most common reaction by AI researchers to these prophecies of doom is face palming." to be lacking in awareness. "Experts disagree on risk of extinction" isn't quite as reassuring as he thinks it is. The reality is, despite the opinions of the armchair quarterbacks commentin…

> I find it quite extraordinary how many on here are dismissing that there is any risk at all. The fear over AI is a displaced fear of unaccountable social structures with extinction-power that currently exist and we allow to continually exist . Without these structures AI is harmless to the species, even superintelligence. Your (reasonable) counter-argument might be that somebody (like, say, my dumb self) accidental…

> One example is the Outer Space Weapons Ban treaty - we can have a similar ban of AI in militaries

It's extremely unclear, in fact, whether such a ban would be enforceable.

Detecting outer space weapons is easy. Detecting whether a country is running advanced AIs in their datacenter is a lot harder.

Post reply on HN