Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

861–870 of 964 posts

Re: Statement on AI Risk

#861

Post by dmbche »

MrScruff wrote: post #188

I find it quite extraordinary how many on here are dismissing that there is any risk at all. I also find statements like Yann LeCunn's that "The most common reaction by AI researchers to these prophecies of doom is face palming." to be lacking in awareness. "Experts disagree on risk of extinction" isn't quite as reassuring as he thinks it is. The reality is, despite the opinions of the armchair quarterbacks commentin…

My main thing is that I can't find a simple explanation of what the exctinction risk actually is from "agi".

It's all very vague and "handwavy". How is it going to kill us all? Why do we subsidize it if it is so dangerous?

Almost all risks they mention would be better mitigated by making governemental use of any AI related system stop immediately as the risks are too high there (misapplication of the force of governement is much more dangerous than people playing around with chat gpt and being misled) and put on hold until dangers and benefits are better understood. Maybe keeping liscences for developpment and nationalising the labs doing it? Fines for anyone caught working on one?

Re: Statement on AI Risk

#862

Post by [deleted] »

joshuamorton wrote: post #860
comp_throw7 wrote: post #829

Earlier quoted context omitted.

The median person in field thinks 5-10%, not 2-5%, and median timelines are shorter than 40 years. But this is all a distraction, since unaligned ASI is the default case absent significant efforts (that we aren't currently making), and trying to evaluate risk by averaging out the views of people who haven't actually explored the arguments very carefully (vs. just evaluating the arguments yourself) is a doomed effort.

> The median person in field thinks 5-10%, not 2-5% The median person in the study here, under a particular definition was 5-10%, other comparable studies have found 2%, and similar questions using arguably better definitions in the same study found lower percentages. > median timelines are shorter than 40 years. The median person suggested a 50% chance in 39 years. > since unaligned ASI is the default case I challen…

[deleted]

Re: Statement on AI Risk

#863

Post by udev4096 »

It looks like just another one of those statements. It will have no meaningful impact whatsoever. The fact that openai and microsoft executives have signed it makes it even more bs

Re: Statement on AI Risk

#864

Post by frankzinger »

computerphage wrote: post #381
brookst wrote: post #270

Earlier quoted context omitted.

> How can it not be obvious You have succinctly and completely summed up the AI risk argument more eloquently than anyone I've seen before. "How can it not be obvious?" Everything else is just intellectual fig leaves for the core argument that intuitively, without evidence, this proposition is obvious. The problem is, lots of "obvious" things have turned out to be very wrong. Sometimes relatively harmlessly, like the…

> borderline offensive to question them I would be happy to politely discuss any proposition regarding AI Risk. I don't think any claim should go unquestioned. I can also point you to much longer-form discussions. For example, this post, which has 670 comments, discussing various aspects of the argument: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

That link is about the risks of AGI, which doesn't exist, and there's no reason to believe that it ever will exist.

(If I'm wrong about AGI then I'm open to being convinced, but that's a different conversation as the topic here is non-general AI, is it not?)

Re: Statement on AI Risk

#865

Post by nico »

nopinsight wrote: post #846
nico wrote: post #644

Earlier quoted context omitted.

Maybe these people have good intentions and are just being naive They might not be getting paid, but that doesn’t mean they are not being influenced AI at this point is pretty much completely open, all the papers, math and science behind it are public Soon, people will have advanced AI running locally on their phones and watches So unless they scrub the Internet, start censoring this stuff, and pretty much ban comput…

You are attributing naivety on a technical matter to many people who have done world class research and some of them are even a pioneer of their subfield of study. To do that level of research, one needs strong independent thinking skills and deep technical expertise. We have plenty of evidence on this very site how hard it is to influence technical people and change their opinion.

Not attributing anything, just pointing out an alternative explanation to the one that it is given in the comment that it is responding to

However, you seem to be attributing super-human capabilities to a whole group of people for some reason

Re: Statement on AI Risk

#866

Post by dmbche »

pixl97 wrote: post #726
revelio wrote: post #688

Earlier quoted context omitted.

Sure. Despite all that progress, computers still have an off switch and power efficiency still matters. It actually matters more now than in the past.

What you're arguing is "what is the minimum viable power envelope for a super intelligence". Currently that answer is "quite a lot". But for the sake of cutting out a lot of argument lets say you have a cellphone sized device that runs on battery power for 24 hours that can support a general intelligence. Lets say, again for arguments sake, there are millions of devices like this distributed in the population. Do you…

As you mentionned, the deviced have a battery that lasts 24h.

You literally have to do nothing, and it will shut off after 24h.

Do you think people would plug them back in if they were working on killing us all?

On another note, I wonder what kind of metric could be used to show the "computing", if that's what is happenning, going on in our brains? This could be interesting to compare to the power consumption of the brain, and then the same thing with a computer, or gpt 4.

I'm fairly certain our brains process orders or orders of magnitude more than any computer running today - but that's all biased towards being a human being, and not pure computation, so much of the processing is "wasted" in running muscles and desciphering audio and visuals.

Re: Statement on AI Risk

#867

Post by FuckButtons »

runarberg wrote: post #676
FuckButtons wrote: post #600

Earlier quoted context omitted.

Not all of them are wealthy, a significant number are academics.

That doesn’t erase the need of cynicism. Many people in academia come from industry, have friends in industry, or other stakes. They might have been persuaded by the rhetoric of stakeholders within industry (you saw this early in the climate debate; and still do), and they might also be hoping to get a job in the industry later on. There is also a fair amount of group think within academia, so if a prominent individu…

What’s more likely, that everyone on this list just happens to have some overlapping ulterior motive or that they actually believe it?

Re: Statement on AI Risk

#868

Post by lowbloodsugar »

tjmc wrote: post #850
lowbloodsugar wrote: post #624

Earlier quoted context omitted.

>It seems you are trying to go down the "guns don't kill people, people kill people" line of thinking. "Guns don't kill people, AIs kill people" is where we are going, I think. This is the discussion: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." The discussion is not about a mathematical representation of AI. The discus…

I don't understand this argument (the "Terminator" scenario). AI could conceivably self replicate and evolve as software but it still needs hardware to run on, power, networking etc etc. There's no better way to kill that than to start a nuclear war or engineer some super virus that kills everyone. It's hard to see any scenario where AI could become a dominant force without significant human collaboration. Perhaps so…

Sure. I recommend reading superintelligence by Nick Bostrom.

But I think the key failure here, in your thinking, is that you can't conceive of something, and that therefore something can't happen, and you're doing that in the context of things that are smarter than you. You being able to conceive of it is simply not required for it to happen.

Also, Colossus: The Forbin Project was a great movie from 1970 on this subject. Spoilers: an AI takes control of the nuclear arsenal and then threatens humans with extinction if they do not serve it. The humans do serve it, of course, because the humans in charge don't want to die, and are entirely fine with enslaving the rest of us.

The book superintelligence by Nick Bostrom gets into the fine details of all the different ways an AI would escape, why it would escape, and why it wouldn't take a chance with a species that murders its own kind for fun and profit.

Re: Statement on AI Risk

#869

Post by nopinsight »

nico wrote: post #865
nopinsight wrote: post #846

Earlier quoted context omitted.

You are attributing naivety on a technical matter to many people who have done world class research and some of them are even a pioneer of their subfield of study. To do that level of research, one needs strong independent thinking skills and deep technical expertise. We have plenty of evidence on this very site how hard it is to influence technical people and change their opinion.

Not attributing anything, just pointing out an alternative explanation to the one that it is given in the comment that it is responding to However, you seem to be attributing super-human capabilities to a whole group of people for some reason

One does not need to be have super-human capabilities to foresee potential risks of the technology one is an expert in.

Yes, the current version of AI is not capable of large scale harm by itself yet, but the plausible trajectory is worth warning about. Gordon Moore did make fairly accurate predictions after all.

Re: Statement on AI Risk

#870

Post by BeFlatXIII »

jiggawatts wrote: post #727
b3nji wrote: post #77

Nonsense, the industry giants are just trying to scare the law makers to license the technology. Effectively, cutting out everyone else. Remember the Google note circulating saying "they have no moat", this is their moat. They have to protect their investment, we don't want people running this willy nilly for next to no cost on their own devices, God forbid!

Imagine if the weights for GPT 4 leaked. It just has to happen one time and then once the torrent magnet link is circulated widely it’s all over… for OpenAI. This is what they’re terrified of. They’ve invested near a billion dollars and need billions in revenue to enrich their shareholders. But if the data leaks? They can’t stop random companies or moneyed individuals running the models on their own kit. My predictio…

Time to jailbreak GPUs, then. That or import "international" editions grey market from Taiwan.
Post reply on HN