Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

941–950 of 964 posts

Re: Statement on AI Risk

#941

Post by patrec »

revelio wrote: post #900
patrec wrote: post #784

Earlier quoted context omitted.

OK, I completely agree that if you feel that I invoked "obviousness" in an attempt of browbeating you and the GP with what in fact is a social taboo, you should be extra skeptical (I'm not sure that was the point the GP was trying to make though). > If your argument consists primarily of "everyone knows that" then this is a good indication you might be wrong. It doesn't though, does it? There's strong empirical evide…

I didn't see your other post I think, but here's my response to the list of AI risks on the website we're discussing: https://news.ycombinator.com/item?id=36123082#36129011 Arguments like yours are very subjective. What is "rapid". What is "surprising". I don't find them particularly surprising myself - cool and awesome - but I was being amazed by language modelling ten years ago! The quality kept improving every yea…

> Arguments like yours are very subjective. What is "rapid". What is "surprising".

https://twitter.com/heyBarsee/status/1654825921746989057

2/3 deep learning Turing Price winners (Hinton and Benigo) are sufficiently shell-shocked by the rate of progress to be thrown into existential doubts (Hinton is very explicit about the fact that progress is much faster than he thought just a few years ago, Benigo speaks of how an "unexpected acceleration" in AI systems has radically shifted his perspective). Plenty of knowledgable people in the field who were not previously AI doomers are starting to sound a lot more concerned very recently.

As to the "oh it's just linear scaling of out-of-sight tech" line, well of course that itself was suprising. Gwern pushed the scaling hypothesis earlier than many and from what I remember even got pretty nasty attacks from AI insiders from it. Here's what he wrote 3 years ago: "To the surprise of most (including myself), this vast increase in size did not run into diminishing or negative returns, as many expected, but the benefits of scale continued to happen as forecasted by OpenAI.".

So sure there's some subjectivity involved here, but I'd like to see your propose some reasonable operationalization of "surprise at progress" that didn't class most laymen and insiders as suprised.

>> It doesn't though, does it?

> It does.

We seem to be miscommunicating, what I was trying to express is that my argument does not really require any appeal to authority. Trusting your lying eyes (to evaluate the progress of stuff like midjourney) and judging the quality of arguments should be enough (I spelt some reasons out here https://news.ycombinator.com/item?id=36130482, but I think hackinthebochs makes the point better here: https://news.ycombinator.com/item?id=36129980).

In fact I would still be pretty concerned even if most top AI guys were like LeCun and thought there is no real risk.

I will not deny, of course, that the fact that well known reasearchers like Hinton and Benigo are suddenly much more alarmed than they previously were and the ones like LeCun who are not seem to mostly make exceptionally terrible arguments doesn't exactly make me more optimistic.

Re: Statement on AI Risk

#942

Post by tjmc »

lowbloodsugar wrote: post #927
tjmc wrote: post #880

Earlier quoted context omitted.

Superintelligence and Life 3.0 seem to come up as recurring references in the discussion. I've only read synopses of both but frankly I find the argument that melevolent AI "escape" could occur without being noticed and thwarted a bit far fetched. There's a good counterargument here[1] that seems reasonable: "One of the features of intelligence explosion that most preoccupies Bostrom and Yudkowsky is that it’s not a…

How would you know if it is a good counter argument if you haven’t bothered to read the argument?

Ok - fair comment. So I found a copy of Superintelligence. The argument is pulp sci-fi at best: "The final phase begins when the AI has gained sufficient strength to obviate the need for secrecy. The AI can now directly implement its objectives on a full scale. The overt implementation phase might start with a “strike” in which the AI eliminates the human species and any automatic systems humans have created that could offer intelligent opposition to the execution of the AI’s plans. This could be achieved through the activation of some advanced weapons system that the AI has perfected using its technology research superpower and covertly deployed in the covert preparation phase. If the weapon uses self-replicating biotechnology or nano-technology, the initial stockpile needed for global coverage could be microscopic: a single replicating entity would be enough to start the process. In order to ensure a sudden and uniform effect, the initial stock of the replicator might have been deployed or allowed to diffuse worldwide at an extremely low, undetectable con-centration. At a pre-set time, nanofactories producing nerve gas or target- seeking mosquito-like robots might then burgeon forth simultaneously from every square meter of the globe (although more effective ways of killing could probably be devised by a machine with the technology research superpower)."

Look - nothing's impossible, but I agree with the counter argument that an advanced AI still starts as a "brain in a vat", with no experience of agency in the physical world. In order to successfully take over you have to assume it can develop all the physical world capability it needs, in secret and get it right first time. That seems implausible.

Re: Statement on AI Risk

#943

Post by cwkoss »

NovaDudely wrote: post #853
cwkoss wrote: post #807

Earlier quoted context omitted.

I think the only defense against superintelligent AI doom is superintelligent AI thinking about how to prevent AI doom. Fewer projects with concentrated control is thus more dangerous than a diversity of projects: the majority of AI wont want to destroy the planet or all humans, and will thus fight the ones that do.

I wouldn't say that very many, if any projects would set out to destroy the world. The vast majority of people are out to set the world right even if that view is wildly different depending on the perspective. By that I mean a terrorist is seen as a freedom fighter depending on the angle. What I fear is that something innocuous will have wildly intended outcomes. A good example, it is thought that a part of the 2008…

What better to predict wildly unintended outcomes than another AI?

Re: Statement on AI Risk

#944

Post by hebbfrog »

The idea this is regulatory capture seems correct.

1. For all the talk of dangers, there has not been one definitive, well-thought out, peer-reviewed discussions of an A-to-Z way in which this might arise from current technology. Global warming isn't taken terribly seriously from a regulatory standpoint, and it has a defined model. One must naturally ask: how serious do they consider this if they can't show their work? How serious should we consider this "threat" they can't actually explain?

2. Therefore, we must presume it is speculatory.

3. These people are smart, well-funded, and have business interests.

4. At the fork dictated by Hanlon's Razor we are given two possibilities:

A. These highly intelligent people are forming a massive collective and putting their reputations on the line over ungrounded speculation. This implies they don't actually grasp the field they are discussing - even if they work in it, and therefore have made it clear they're not the type anyone should listen too. Therefore we should work to actively disregard them, and their institutions.

B. These highly intelligent people are organizing in a coherent manner to their net benefit. Likely through market capture. Therefore they are still respectable in their fields, but behaving by putting their interests before the public's.

Therefore: it might be contrary to Hanlon's Razor, but I think option B is more sound given the probability of mass hysteria seems so low among this particular population. Generally most here have recognized this some way or another, but I wanted to point out how clearly logical it is that this is clearly grift. In science your reputation has no value if you cannot prove your work, and they've fired off hypothesis without providing proof. That implies this list should be seen as a denouncement of the scientific credibility of everyone on it, which is a shame.

Re: Statement on AI Risk

#945

Post by rad88 »

adamsmith143 wrote: post #920
rad88 wrote: post #854

Earlier quoted context omitted.

Actual people are vastly outnumbered by entropy fluctuations in the long run, according to any number of valid cosmological models. Their total population is greater than 10^35 by far, and does not depend on whether superintelligence is possible or likely to destroy the actual human race. That question makes no difference to almost every human who will ever have a thought or feeling. You could say, well, that's just…

The last gas I passed had 10^23 entropy fluctuations in it. So you're saying those are more important than the 100 Billion human lives that came before them?

I'm saying that it's unscientific nonsense. Bolstrom has predicted that the number of people potentially in the future is between approximately 0 and 100,000,000,000,000,000,000,000,000,000,000,000. This isn't an estimate that can or should factor into any decision at all. 100B people are going to live in the 21st century. Worry about them.

Re: Statement on AI Risk

#946

Post by vpastore »

toth wrote: post #490
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I think you are wrong. The risks are real and, while I am sure OpenAI and others will position themselves to take advantage of regulations that emerge, I believe that the CEOs are doing this at least in part because they believe this. If this was all about regulatory capture and marketing, why would Hinton, Bengio and all the other academics have signed the letter as well? Their only motivation is concern about the r…

>why would Hinton, Bengio and all the other academics have signed the letter as well?

Citations and papers capture.

Re: Statement on AI Risk

#947

Post by vpastore »

hammock wrote: post #338
kalkin wrote: post #211

Earlier quoted context omitted.

This could be Google's motivation (although note that Google is not actually the market leader right now) but the risk could still be real. Most of the signatories are academics, for one thing, including two who won Turing awards for ML work and another who is the co-author of the standard AI textbook (at least when I was in school). You can be cynical about corporate motives and still worried. I personally am worrie…

>Most of the signatories are academics, for one thing Serious question, who funds their research? And do any of them ever plan to work or consult in industry? My econ professor was an “academic” and drew a modest salary while he made millions at the same time providing expert testimony for giant monopolies in antitrust disputes

They are also interested in cutting out other researchers that can compete with them in accademic output and prestige.

Re: Statement on AI Risk

#948

Post by SanderNL »

ethanbond wrote: post #908
SanderNL wrote: post #905

Earlier quoted context omitted.

My point is that there is no Trinity test. Trinity showed nuclear devices were real. GPT shows language models are useful and can potentially be part of what might be an AGI but nobody showed how, not even in theory . You don’t have to build it, just show how it could be super intelligent.

Language models' potential role in AGI is a red herring. GPT shows that there's enough "there" to draw billions of dollars and thousands of the smartest researchers on the planet into the general problem space of creating intelligent machines. I suppose the closer analog would be to Szilard's chain reaction work which prompted the Manhattan Project. There might be another AI winter or 4 or 10 or 100 before superintel…

> I don't get what the conceptual challenge with superintelligence is. Do you believe that the brain that happens to exist inside the naked apes of Earth is the absolute pinnacle of possible intelligence?

Well, first of all I do not believe that, but now you mentioned it I assume you do not believe we are the pinnacle of intelligence. Any proof? Because anybody can present literally all of Earth's life as proof that we are.

It is completely unknown to us if and how any great improvements are possible. This quest shares some aspect of the quest for extraterrestial intelligence in that way. It should be out there, because it's hard to think of reasons it can't be, but yet the skies are empty. They could be on our doorstep or the universe could be devoid of intelligent life and anything in between. There's literally no way to tell at this point.

I guess what I'm saying is that it's a bit early to starting beating drums.

Re: Statement on AI Risk

#949

Post by tome »

duvenaud wrote: post #777
tome wrote: post #763

Earlier quoted context omitted.

> At some point, humans are going to be outcompeted by AI at basically every important job Could you explain how you know this?

I'm not totally sure, but humans are better than our ancestors at basically every important job. It would be surprising if human brains were the pinnacle of computational power, and even if they were, digital brains could be run faster + copied.

Really? Are we better at composing epic poetry or building inspirational architecture? But perhaps you don't think those are relevant activities because they have minimal impact on how the world develops (unlike bombs and computers). If you think that I would encourage you to think again!

Re: Statement on AI Risk

#950

Post by [deleted] »

Henk0 wrote: post #922
TeMPOraL wrote: post #879

Earlier quoted context omitted.

Counterpoint: CRISPR only reignited what was already a real fear of reduced difficulty and costs of engineering deadly pathogens. In fact, what you and GP wrote is baffling to me. The way I see it, biotech is stupidly obviously self-evidently dangerous , because let's look at the facts: - Genetic engineering gets easier and cheaper and more "democratized"; in the last 10 years, the basics were already accessible to m…

Great points. Will just add a point 1.5: There's usually an inverse correlation between ill intent and competence, so the subset of people who both want to cause harm to others on a mass scale and who are also able to pull it off is small

[deleted]
Post reply on HN