Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Have top AI research institutions just given up on the idea of safety?

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 99 posts

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#81

Post by scarmig »

sigbottle wrote: post #79
SpicyLemonZest wrote: post #48

Earlier quoted context omitted.

In the context of AI research, there is no question that "existential" means "powerful AI literally kills every human being". It's a mainstream although not universal view among experts in the space that this is a serious possibility.

That's not my point. My point is the moralizing and worshipping around it. For example - by powerful, do you mean a mass government surveillance system? That can be implemented by AI of today right now, even if AI stagnated. It's the argument where oh, AI is just a superset of all humans, humans are dumb and don't even know themselves, we should just submit esque attitude that I'm talking about. The easiest way to so…

At least in the case of the researchers I mentioned, they have a deeply held, genuine belief that AI will, in the very near term, exceed humans in all intellectual capabilities, and that poses a bigger risk to human existence than humans simply fucking things up (beyond the fuck up of competently building a superior being). I would bet that most of them believe that us being paperclipped is a more likely bad outcome than a dystopia arising from human control. Simply because a human dystopia takes time to implement, even when aided by AI, which is time we don't have.

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#82

Post by SpicyLemonZest »

sigbottle wrote: post #79
SpicyLemonZest wrote: post #48

Earlier quoted context omitted.

In the context of AI research, there is no question that "existential" means "powerful AI literally kills every human being". It's a mainstream although not universal view among experts in the space that this is a serious possibility.

That's not my point. My point is the moralizing and worshipping around it. For example - by powerful, do you mean a mass government surveillance system? That can be implemented by AI of today right now, even if AI stagnated. It's the argument where oh, AI is just a superset of all humans, humans are dumb and don't even know themselves, we should just submit esque attitude that I'm talking about. The easiest way to so…

By "powerful", I mean a system whose operations humans cannot control or prevent or even reason about, in the same way that the members of an anthill can't do anything about a construction crew dumping concrete on them to lay a sidewalk. It's got nothing to do with "should submit" or "morally deserves". If the AI system in question is capable enough, it simply won't matter any longer what any human being thinks should happen. (In principle, it also has to be autonomous; in practice, I think OpenClaw has clearly illustrated that any AI system is going to be granted autonomy by someone.)

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#83

Post by scarmig »

helloplanets wrote: post #69
scarmig wrote: post #11

More on the periphery than an insider, but I personally know researchers in all three major labs who were there long before GPT-3. They all care about existential safety, a lot. In the sense that they believe there's a meaningful chance all humans are dead a decade from now (and that that's a bad thing; unfortunately, there are also people deeply involved who don't think human extinction is a bad thing). The issue is…

> unfortunately, there are also people deeply involved who don't think human extinction is a bad thing You mean at the top labs? Since when isn't that level of misanthropy categorized as having mental health issues?

See e.g. Richard Sutton, who, although not at a top lab, is certainly a very important figure in the field.

Or, if you want someone with concrete influence at a top lab, Larry Page.

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#84

Post by AndrewKemendo »

chasd00 wrote: post #80
AndrewKemendo wrote: post #68

Earlier quoted context omitted.

The only “safe AI” is one that comes out of a “safe set of data” so what would a “safe set of data” actually have to look like Well it would have to not look like the majority of data that we produce now which has latent embeddings (primarily from the common crawl database ) of racism, lying, competition, destruction domination I don’t believe humans are actually capable of making such data because our entire structu…

> has latent embeddings (primarily from the common crawl database ) of racism, lying, competition, destruction domination but safety has a wider scope than "racism, lying, competition, destruction domination" like always requiring eye protection when asked about making lemonaide. > I don’t believe humans are actually capable of making such data because our entire structure of society is based on racism competition an…

Indeed that conclusion was reached a long time ago but technologists literally don’t care because they’re just trying to get paid

And literally everybody who has been trying to warn about it is beaten down publicly as a radical or whatever

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#86

Post by kjkjadksj »

davidguetta wrote: post #85

what is "safe". models don't have hands to hurt me

The people prompting models do. I saw a job post a while back at RAND. It was for researchers with experience in bioweapons design to see how far along current models are for a motivated person to cause serious harm using them. I believe they have a number of publications now on this effort.

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#87

Post by neko_ranger »

Alignment of AI is hard, and aligned to whom? I just finished the safety chapter in Stripe Press's "An Oral History of AI", and there's a good quote in it: "It's an interesting question, how to tell the difference between a hallucination and deception." (I'll let you figure out who said it, you know their name).

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#88

Post by terminalshort »

Sharlin wrote: post #67
terminalshort wrote: post #56

Earlier quoted context omitted.

If we had rules like that in the past we never would have had the industrial revolution.

Yes, it would probably have been better to have industrial evolution instead. Or are you arguing that all the countless deaths, maimings, child labor, 16-hour workdays, robber barons, black lung, radium jaws, and so on and so on were simply how it had to go? Or do you simply not care because all of that happened to other people?

Yeah, pretty much. A material emitting a previously unknown form of energy that turns out to be extremely harmful is really something you can only discover by trial and error. And what do you mean it's happening to other people? I am being exposed to all kinds of shit like PFAs and microplastics today. But it turns out that the technological progress outweighs all the environmental pollutants and accidents that it took to get here and we still live healthier and longer lives than we did before.

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#89

Post by terminalshort »

program_whiz wrote: post #62
terminalshort wrote: post #56

Earlier quoted context omitted.

If we had rules like that in the past we never would have had the industrial revolution.

Not sure if that's true, what are your reasons for believing that? Are you saying we couldn't have invented the machines we used if we took safety measures along the way (e.g. having guards on machines that chopped of arms and legs)? Perhaps progress would have been slower -- since rather than just using the saw, you'd need a saw with a guard and emergency switch -- but it seems like if humans were more circumspect,…

And progress being slower means people die. You ever had polio? Smallpox? When was the last famine you lived through?

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#90

Post by ASalazarMX »

WarmWash wrote: post #47

Safety means slower and this is viewed as a winner takes all game. This isn't new either, the safety glass cracked the day OpenAI publicly launched ChatGPT. "Safety" was (and perhaps still is) a fall back for the models plateauing and LLMs failing to really make an impact..."we need more time while we focus on safety" But after this latest round of models, it's a lot more fuel on the "this could be it" fire. Labs are…

Research was being done slower, until OpenAI forced everyone to jump the gun or potentially be left behind. For a few months it looked like everyone was light years behind them.
Post reply on HN