Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

221–230 of 964 posts

Re: Statement on AI Risk

#221

Post by patrec »

valine wrote: post #140
patrec wrote: post #58

Earlier quoted context omitted.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

It's not clear at all that we have an avenue to super intelligence. I think the most likely outcome is that we hit a local maximum with our current architectures and end up with helpful assistants similar in capability to George Lucas's C3PO. The scary doomsday scenarios aren't possible without an AI that's capable of both strategic thinking and long term planning. Those two things also happen to be the biggest limit…

> It's not clear at all that we have an avenue to super intelligence

AI already beats the average human on pretty much any task people have put time into, often by a very wide margin and we are still seeing exponential progress that even the experts can't really explain, but yes, it is possible this is a local maximum and the curve will become much flatter again.

But the absence of any visible fundamental limit on further progress (or can you name one?) coupled with the fact that we have yet barely begun to feel the consequences of the tech we already have (assuming zero breakthroughs from now on) makes we extremely wary to conclude that there is no significant danger and we have nothing to worry about.

Let's set aside the if and when of a super intelligence explosion for now. We are ourselves an existence proof of some lower bound of intelligence, that if amplified by what computers can already do (like perform many of the things we used to take intellectual pride in much better, and many orders of magnitude faster with almost infinitely better replication and coordination ability) seems already plenty dangerous and scary to me.

> The scary doomsday scenarios aren't possible without an AI that's capable of both strategic thinking and long term planning. Those two things also happen to be the biggest limitations of our most powerful language models. We simply don't know how to build a system like that.

Why do you think AI models will be unable to plan or strategize? Last I checked languages models weren't trained or developed to beat humans in strategic decision making, but humans already aren't doing too hot right now in games of adversarial strategy against AIs developed for that domain.

Re: Statement on AI Risk

#222

Post by lb4r »

endisneigh wrote: post #191
lb4r wrote: post #183

Earlier quoted context omitted.

I think if you were referring to AI in its 'current form' all along, then most people will probably agree with you, myself included. But 20 years from now? I personally think it would be arrogant to dismiss the potential dangers.

if we are talking about regulating something now, we must talk about capabilities now. there's no point in talking about nonexistent technology. should we also regulate teleportation? it's been done in a lab. if AI actually is a threat, then it can be regulated. it's not a threat now, period. preemptively regulating something is silly and a waste of energy and political capital.

You added the paragraph about regulation after I had written my comment to your initial post, so I was really only talking about what I initially quoted. The question about regulation is complex and something I personally have yet to make up my mind about.

Re: Statement on AI Risk

#223

Post by sanderjd »

rockemsockem wrote: post #186
sanderjd wrote: post #158

Earlier quoted context omitted.

I agree that the second question is way more interesting, and I'm glad there's a lot of ongoing discussion and debate about it. And you have some insightful thoughts on it here. But I disagree with you that this is clearly what the NYT article is about. There is a significant focus on the "industry leaders" who have been most visible in - suddenly! - pushing for regulation. And that's why people are reasonably pointi…

Cite where it is being said that these companies are arguing "to make their models legal, but any competitive upstarts illegal". As far as I know nothing of the sort has been proposed. You may think this is obvious, but it is far from it.

That's what the "AI pause" proposed. But the main thing is: they could shut down their current technology themselves, so what they are arguing for must be regulation of future technology. I think this has been pretty clear in the congressional hearings for instance.

Re: Statement on AI Risk

#224

Post by 0xbadc0de5 »

Two things can be true - AI could someday pose a serious risk, and anything the current group of "Thought Leaders" and politicians come up with will produce a net-negative result.

Re: Statement on AI Risk

#225

Post by duvenaud »

DirkH wrote: post #218
jacurtis wrote: post #129

Reading into the early comments from this piece, there is a clear divide in opinions even here on HN. The opinions seem to fall into two camps: 1) This is just a move that evil tech companies are making in order to control who has access to AI and to maintain their dominance 2) AI is scary af and we are at a inflection point in history where we need to proceed cautiously. This NYTimes piece is clearly debating the la…

Worth adding that there is no contradiction in strongly believing both 1+2 are true at the same time. I.e. Evil tech companies are just trying to maintain their control and market dominance and don't actually care or think much about AI safety, but that we are nonetheless at an inflection point in history because AI will become more and more scary AF. It is totally plausible that evil tech got wind of AI Safety conce…

AI Safety hasn't been divorced from tech companies, at least not from Deepmind, OpenAI, and Anthropic. They were all founded by people who said explicitly that AGI will probably mean the end of human civilization as we know it.

All three of them have also hired heavily from the academic AI safety researcher pool. Whether they ultimately make costly sacrifices in the name of safety remains to be seen (although Anthropic did this already when they delayed the release of Claude until after ChatGPT came out). But they're not exactly "watching from the sidelines", except for Google and Meta.

Re: Statement on AI Risk

#226

Post by somewhereoutth »

dwaltrip wrote: post #198
sf4lifer wrote: post #139

LLMs are just text prediction. I don't see the linear path from LLM to AGI. Was there similar hysteria when the calculator or PC first came out?

It’s not just random text, they are predicting writings and documents produced by humans. They are “language” models. Language is used to say things about the world. This means that predicting language extremely well is best done through acquiring an understanding of the world. Take a rigorous, well-written textbook. Predicting a textbook is like writing a textbook. To write a good textbook, you need to be an expert…

To be more precise, it holds a model of the text that has been fed to it. That will be, at very best, a pale reflection of the underlying model of the world.

Re: Statement on AI Risk

#227

Post by Infernal »

duvenaud wrote: post #209
skepticATX wrote: post #72

Earlier quoted context omitted.

But what argument is there to refute? It feels like Aquinas “proving” God’s existence by stating that it is self evident. They can’t point to an existing system that poses existential risk, because it doesn’t exist. They can’t point to a clear architecture for such a system, because we don’t know how to build it. So again, what can be refuted?

Here's one of my concrete worries: At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an out-of-touch old person on the internet - we'll either have to delegate everything important (which is risky), or eventually get scammed or extorted out of all our resources and influenc…

“Humans are going to be outcompeted by AI” is the concrete bit as best I can tell.

Historically humans are not outcompeted by new tools, but humans using old tools are outcompeted by humans using new tools. It’s not “all humans vs the new tool”, as the tool has no agency.

If you meant “humans using old tools get outcompeted by humans using AI”, then I agree but I don’t see it any differently than previous efficiency improvements with new tooling.

If you meant ”all humans get outcompeted by AI”, then I think you have a lot of work to do to demonstrate how AI is going to replace humans in “every important job”, and not simply replace some of the tools in the humans’ toolbox.

Re: Statement on AI Risk

#228

Post by patrec »

randomdata wrote: post #123
patrec wrote: post #58

Earlier quoted context omitted.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

Nuclear missiles present an obvious danger to the human body. AI is an application of math. It is not clear how that can be used directly to harm a body. The assumption seems to be that said math will be coupled with something like a nuclear missile, but in that case the nuclear missile is still the threat. Any use of AI is just an implementation detail.

I'm glad to learn that Hitler and Stalin were both "implementation details" and not in any way threatening to anyone.

Re: Statement on AI Risk

#229

Post by pixl97 »

a_bonobo wrote: post #59
sebzim4500 wrote: post #55

Earlier quoted context omitted.

In what sense is this a 'biased statement' exactly? If a dozen of the top climate scientists put out a statement saying that fighting climate change should be a serious priority (even if they can't agree on one easy solution) would that also be 'biased'?

Climate change is a generally accepted phenomenon. Extinction risk due to AI is not a generally accepted phenomenon.

>Extinction risk due to AI is not a generally accepted phenomenon

Why?

You, as a species, are the pinnacle of NI, natural intelligence. And with this power that we've been given we've driven the majority of large species, and countless smaller species to extinction.

To think it outside the realms of possibility that we could develop an artificial species that is more intelligent than us is bizarre to me. It would be like saying "We cannot develop a plane that does X better than a bird, because birds are the pinnacle of natural flying evolution".

Intelligence is a meta-tool, it is the tool that drives tools. Humanity succeeded above all other species because of its tool using ability. And now many of us are hell bent on creating ever more powerful tool using intelligences. To believe there is no risk here is odd in my eyes.

Re: Statement on AI Risk

#230

Post by thrillgore »

If there's anyone who can speak to the risk of AI, its Sam Altman, the signatory of this letter, CEO of OpenAI, a member of Y Combinator, and a contributor to Hacker News.

Instead of making this a diversionary puff piece, I would like to hear Sam provide tangible feedback on how we can mitigate the risks AI bring us, since he is the one that started the AI revolution.

Post reply on HN