Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

411–420 of 964 posts

Re: Statement on AI Risk

#411

Post by mrtranscendence »

olalonde wrote: post #352
brookst wrote: post #259

Earlier quoted context omitted.

Odd that you lump a person's choice of their own pronouns into a legitimate complaint that all she seems to have is prejudice and ad hominems.

I have no issue with her choice of pronouns. I just find it odd that she states them when ~100% of the population would infer them correctly from her name Emily (and picture). My guess is she put them there for ideological signaling.

This is unnecessarily cynical. Why should people who are androgynous or trans be the only ones who state pronouns? By creating a norm around it we can improve their comfort at extremely minimal cost to ourselves.

Re: Statement on AI Risk

#412

Post by RandomLensman »

holmesworcester wrote: post #378
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

As others have pointed out, there are many on this list (Bruce Schneier, for example) who do not stand to benefit from AI marketing or regulatory capture. Anyone upvoting this comment should take a long look at the names on this letter and realize that many are not conflicted. Many signers of this letter are more politically sophisticated than the average HN commenter, also. So sure, maybe they're getting rolled by m…

Who on the list is an expert on existential risk (and perhaps even beyond academia)?

Re: Statement on AI Risk

#413

Post by boringuser2 »

A lot of people in this thread seem to be suffering from a lack of compute.

The idea that an AI can't be dangerous because it is a incorporeal entity trapped in electricity is particularly dumb.

This is literally how your brain works.

You didn't build your house. You farmed the work out using leverage to people with the skills and materials. Your leverage was meager wealth generated by a loan.

The leverage of a superintellect would eclipse this.

Re: Statement on AI Risk

#414

Post by pixl97 »

RandomLensman wrote: post #369
pixl97 wrote: post #213

Earlier quoted context omitted.

>It's not clear at all that we have an avenue to super intelligence. All problems in reality are probability problems. If we don't have a path to superintelligence, then the worst problems just don't manifest themselves. If we do have a path to super intelligence then the doomsday scenarios are nearly a certainty. It's not really any different than saying "A supervolcano is unlikely to go off tomorrow, but if a super…

Why are doomsday scenarios are certainty then. What's the model to get to that that isn't just some sort of scary story that waves away or into existence a lot of things we don't know if they can exist.

>What's the model to get to that

Let's say I was a small furry mammal that tasted really good, but also for some reason understood the world as it is now.

I would tell you that super intelligence had already happened. That super intelligence was humans. That humans happened to reach super intelligence by 1) having the proper hardware. 2) filtering noise from important information. 3) then sharing that information with others to amplify the power of intelligence 4) having a toolkit/tools to turn that information into useful things. 5) And with all that power humans can kill me off in mass, or farm me for my tasty meat at their leisure with little to nothing that I can do about it.

There doesn't appear to be any more magic than that. All these things already exist in biological systems that elevated humans far above their warm blooded peers. When we look at digital systems we see they are designed to communicate. You don't have an ethernet jack as a person. You can't speak the protocol to directly drive a 3 axis mill to produce something. Writing computer code is a pain in the ass to most of us. We are developing a universal communication intelligence, that at least in theory can drive tools at a much higher efficiency than humans will ever be able to.

Coming back to point 5. Cats/dogs are the real smart ones here when dealing with superintelligences. Get domesticated by the intelligence so they want to keep you around as a pet.

Re: Statement on AI Risk

#415

Post by holmesworcester »

hammock wrote: post #338
kalkin wrote: post #211

Earlier quoted context omitted.

This could be Google's motivation (although note that Google is not actually the market leader right now) but the risk could still be real. Most of the signatories are academics, for one thing, including two who won Turing awards for ML work and another who is the co-author of the standard AI textbook (at least when I was in school). You can be cynical about corporate motives and still worried. I personally am worrie…

>Most of the signatories are academics, for one thing Serious question, who funds their research? And do any of them ever plan to work or consult in industry? My econ professor was an “academic” and drew a modest salary while he made millions at the same time providing expert testimony for giant monopolies in antitrust disputes

That's a good question, but at least some of the academics on this list are independent. Bruce Schneier, for example.

Re: Statement on AI Risk

#416

Post by goolulusaurs »

jabradoodle wrote: post #401
goolulusaurs wrote: post #388

Throughout history there have been hundreds, if not thousands of examples of people and groups who thought the end of the world was imminent. So far, 100% of those people have been wrong. The prior should be that the people who believe in AI doomsday scenarios are wrong also, unless and until there is very strong evidence to the contrary. Vague theoretical arguments are not sufficient, as there are many organizations…

What constitutes strong evidence? The obvious counter to your point is that an intelligence explosion would leave you with no time to react.

Well, for example I believe that nukes represent an existential risk, because they have already been used to kill thousands of people in a short period of time. What you are saying doesn't really counter my point at all though, it is another vague theoretical argument.

Re: Statement on AI Risk

#417

Post by adverbly »

goolulusaurs wrote: post #388

Throughout history there have been hundreds, if not thousands of examples of people and groups who thought the end of the world was imminent. So far, 100% of those people have been wrong. The prior should be that the people who believe in AI doomsday scenarios are wrong also, unless and until there is very strong evidence to the contrary. Vague theoretical arguments are not sufficient, as there are many organizations…

Fun! Let me try one:

Throughout history there have been millions, if not billions of examples of lifeforms. So far, 100% of those which are as intelligent as humans have dominated the planet. The prior should be that the people who believe AI will come to dominate the planet are right, unless and until there is very strong evidence to the contrary.

Or... those are both wrong because they're both massive oversimplifications! The reality is we don't have a clue what will happen so we need to prepare for both eventualities, which is exactly what this statement on AI risk is intended to push.

Re: Statement on AI Risk

#418

Post by haswell »

goolulusaurs wrote: post #388

Throughout history there have been hundreds, if not thousands of examples of people and groups who thought the end of the world was imminent. So far, 100% of those people have been wrong. The prior should be that the people who believe in AI doomsday scenarios are wrong also, unless and until there is very strong evidence to the contrary. Vague theoretical arguments are not sufficient, as there are many organizations…

If you were to apply this argument to the development of weapons, it’s clear that there is a threshold that is eventually reached that fundamentally alters the stakes. A point past which all prior assumptions about risk no longer apply.

It also seems very problematic to conclude anything meaningful about AI when realizing that a significant number of those examples are doomsday cults, the very definition of extremist positions.

I get far more concerned when serious people take these concerns seriously, and it’s telling that AI experts are at the forefront of raising these alarms.

And for what it’s worth, the world as many of those groups knew it has in fact ended. It’s just been replaced with what we see before us today. And for all of the technological advancement that didn’t end the world, the state of societies and political systems should be worrisome enough to make us pause and ask just how “ok” things really are.

I’m not an AI doomer, but also think we need to take these concerns seriously. We didn’t take the development of social networks seriously (and continue to fail to do so even with what we now know), and we’re arguably all worse off for it.

Re: Statement on AI Risk

#419

Post by NhanH »

How does one reconcile this with OpenAI claiming they will leave EU if the bloc “over-regulates”.

At extinction level threat, no regulation is over-regulation.

Re: Statement on AI Risk

#420

Post by lxnn »

skepticATX wrote: post #72
apsec112 wrote: post #51

A lot of the responses to this seem like Bulverism, ie., trying to refute an argument by psychoanalyzing the people who argue it: "Suppose I think, after doing my accounts, that I have a large balance at the bank. And suppose you want to find out whether this belief of mine is "wishful thinking." You can never come to any conclusion by examining my psychological condition. Your only chance of finding out is to sit do…

But what argument is there to refute? It feels like Aquinas “proving” God’s existence by stating that it is self evident. They can’t point to an existing system that poses existential risk, because it doesn’t exist. They can’t point to a clear architecture for such a system, because we don’t know how to build it. So again, what can be refuted?

You can't take an empirical approach to existential risk as you don't get the opportunity to learn from your mistakes. You have to prospectively reason about it and plan for it.
Post reply on HN