Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

901–910 of 964 posts

Re: Statement on AI Risk

#901

Post by berniedurfee »

nopinsight wrote: post #642
haldujai wrote: post #632

Earlier quoted context omitted.

> There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. I know the Stanford researchers the most and the “biggest names” in LLMs from HAI and CRFM are absent. It would be useful to have their perspective as well. I’d throw MetaAI in the mix as well. Merely pointing out that healthy skepticism here is not ent…

LLM is already a misnomer. Latest versions are multimodal. Current versions can be used to build agents with limited autonomy. Future versions of LLMs are most likely capable of more independence. Even dumb viruses have caused catastrophic harm. Why? It’s capable of rapid self replication in a massive number of existing vessels. You add in some intelligence, vast store of knowledge, huge bandwidth, and some aid by ma…

“…some intelligence…” appears to be a huge leap from where we seem to be though.

Re: Statement on AI Risk

#902

Post by PoignardAzur »

juve1996 wrote: post #748
PoignardAzur wrote: post #694

Earlier quoted context omitted.

That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". There are lots of situations where offense and defense are asymmetrical. So maybe the killer AI would need two months to build a time-delayed super-virus, and the defender AI would need two months to build a vaccine; if the virus takes less than two months to spread worldwide and ac…

> That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun". Why couldn't it? Metal of X thickness = stopped bullet. Not exactly a hard problem to solve for. Humans managed it quite quickly. But either way it misses the point. > So maybe the killer AI would need two months.. Yes, maybe it would. Maybe it wouldn't. Look at every single one o…

> Why couldn't it? Metal of X thickness = stopped bullet. Not exactly a hard problem to solve for.

I mean... talking about real-world guns, technology already evolved to the point that doesn't really work anymore? Modern body armor isn't about metal of X thickness, it's about material science and spreading out the impacts, and even then it can only sustain a few direct hits.

Like, the asymmetry between offense and defense feels pretty obvious here. You can carry a handgun in a small bag. Bulletproof armor is heavy, cumbersome, conspicuous, etc.

Re: Statement on AI Risk

#903

Post by mahathu »

nopinsight wrote: post #570
chefandy wrote: post #487

Earlier quoted context omitted.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

It’s worth noting also that many academics who signed the statement may face adverse issues like reputational risk as well as funding cut to their research programs if AI safety becomes an official policy. For a large number of them, these risks are worth far more than any possible gain from signing it. When a large number of smart, reputable people, including many with expert knowledge and little or negative incenti…

I think its cringe to both define doom and then come up with some unpredictability precise percentage for that scenario to occur.

Re: Statement on AI Risk

#904

Post by SanderNL »

pixl97 wrote: post #834
tome wrote: post #762

Earlier quoted context omitted.

> lets say you have a cellphone sized device that runs on battery power for 24 hours that can support a general intelligence I can accept that it would be hard to turn off. What I find difficult to accept is that it could exist. What makes you think it could?

You, a general intelligence operate on around 20 watts of power, so we could use that as a base floor. Analog inference is one of the areas being worked on that may massively lower power requirements.

We are also hypermobile generally flexible robots that can run on the occasional pizza.

That doesn’t mean we are anywhere near that level of performance.

Re: Statement on AI Risk

#905

Post by SanderNL »

ethanbond wrote: post #481
SanderNL wrote: post #362

Earlier quoted context omitted.

I understand but agentic/learning general intelligence has not been shown to exist, except ourselves. I’d say this is like worrying about deadly quantum laser weapons that will consume the planet when we are still in the AK47 phase. Edit: it could still be true though. I guess I like some more handholding and pre-chewing before giving governments and large corporations more ropes.

Or it’s like worrying about an arms race toward civilization-ending arsenals after seeing the Trinity test… which… was the correct response. We don’t know it’s possible to build superintelligences but so far we don’t have a good reason to think we can’t and we have complete certainty that humans will spend immense, immense resources getting as close as they can as fast as they can. Very different from the lasers.

My point is that there is no Trinity test. Trinity showed nuclear devices were real. GPT shows language models are useful and can potentially be part of what might be an AGI but nobody showed how, not even in theory.

You don’t have to build it, just show how it could be super intelligent.

Re: Statement on AI Risk

#906

Post by SanderNL »

the8472 wrote: post #474
SanderNL wrote: post #362

Earlier quoted context omitted.

I understand but agentic/learning general intelligence has not been shown to exist, except ourselves. I’d say this is like worrying about deadly quantum laser weapons that will consume the planet when we are still in the AK47 phase. Edit: it could still be true though. I guess I like some more handholding and pre-chewing before giving governments and large corporations more ropes.

The difference to doomsday weapons is that we can build the weapons first and then worry about using them. With an AGI building one alone might be sufficient. It could become smart enough to unbox itself during a training run.

The LHC could have produced a black hole that swallowed the earth, so you’re also too late if that were the case.

Lots of tech is like that, but it turns out we don’t get anything done if we keep fearmongering.

Re: Statement on AI Risk

#907

Post by pixl97 »

tome wrote: post #883
pixl97 wrote: post #834

Earlier quoted context omitted.

You, a general intelligence operate on around 20 watts of power, so we could use that as a base floor. Analog inference is one of the areas being worked on that may massively lower power requirements.

OK, sure, in principle , somewhere in the universe, such a thing could arise. Why do you think there's a path to it arising on planet earth within human timescales?

Because humans keep pouring massive amounts of money into making this happen. When you invest dedicated effort in making something happen you greatly increase the probability that it happens. And in this case it is a reinforcing feedback loop. Better technology begots better technology. Intelligence begots more intelligence.

Evolution has to take a random walk to get where it is, it doesn't necessarily have a goal past continuation, intelligence shortcuts that. It can apply massively iterative efforts towards a goal.

Re: Statement on AI Risk

#908

Post by ethanbond »

SanderNL wrote: post #905
ethanbond wrote: post #481

Earlier quoted context omitted.

Or it’s like worrying about an arms race toward civilization-ending arsenals after seeing the Trinity test… which… was the correct response. We don’t know it’s possible to build superintelligences but so far we don’t have a good reason to think we can’t and we have complete certainty that humans will spend immense, immense resources getting as close as they can as fast as they can. Very different from the lasers.

My point is that there is no Trinity test. Trinity showed nuclear devices were real. GPT shows language models are useful and can potentially be part of what might be an AGI but nobody showed how, not even in theory . You don’t have to build it, just show how it could be super intelligent.

Language models' potential role in AGI is a red herring.

GPT shows that there's enough "there" to draw billions of dollars and thousands of the smartest researchers on the planet into the general problem space of creating intelligent machines. I suppose the closer analog would be to Szilard's chain reaction work which prompted the Manhattan Project.

There might be another AI winter or 4 or 10 or 100 before superintelligence, but they will almost certainly be winters and not ends of the road.

If there is any route to superintelligence and we don't annihilate ourselves by some other method first, we will get there. I don't get what the conceptual challenge with superintelligence is. Do you believe that the brain that happens to exist inside the naked apes of Earth is the absolute pinnacle of possible intelligence?

Re: Statement on AI Risk

#909

Post by brookst »

computerphage wrote: post #381
brookst wrote: post #270

Earlier quoted context omitted.

> How can it not be obvious You have succinctly and completely summed up the AI risk argument more eloquently than anyone I've seen before. "How can it not be obvious?" Everything else is just intellectual fig leaves for the core argument that intuitively, without evidence, this proposition is obvious. The problem is, lots of "obvious" things have turned out to be very wrong. Sometimes relatively harmlessly, like the…

> borderline offensive to question them I would be happy to politely discuss any proposition regarding AI Risk. I don't think any claim should go unquestioned. I can also point you to much longer-form discussions. For example, this post, which has 670 comments, discussing various aspects of the argument: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

I appreciate your reasonableness.

I follow LW to some degree, but even the best of it (like the post you link) feels very in-group confirmation centric.

That post is long and I have not read it all, but it seems to be missing any consideration of AGI upside. It’s like talking about the risk of dying in a car crash with no consideration of the benefits of travel. If I ask you “do you want to get in a metal can that has a small but non-zero chance of killing you”, of course that sounds like a terrible idea.

There is risk in AGI. There is risk in everything. How many people are killed by furniture each year?

I’m not dismissing AGI risk, I’m saying that I have yet to see a considered discussion that includes important context like how many people will live longer/happier because AGI helps reduce famine/disease. Somehow it is always the wealthy, employed, at-risk-of-disruption people who are worried, not the poor or starving or oppressed.

I’m just super not impressed by the AI risk crowd, at least the core one on LW / SSC / etc.

Re: Statement on AI Risk

#910

Post by brookst »

Planktonne wrote: post #335
brookst wrote: post #263

Earlier quoted context omitted.

How are fame, speaking engagements, and book deals not a form of profit? She's intelligent and worth listening to, but she has just as much personal bias and motivation as anyone else.

The (very small) amount of fame she's collected has come through her work in the field, and it's a field she's been in for a while; she's hardly chasing glory.

People don’t have to be chasing fame to be warped by it. She has cultivated a following of like-minded people who provide ever more positive feedback for her ever more ideological writing.

I mean she is literally dismissing people who disagree with her based on their skin color. Can we stop for a minute to wonder about the incentives that encourage that?

(and I generally like her writing and think she has interesting things to say… but I do see a reward cycle going on)

Post reply on HN