Effective Altruism’s obsession with AI safety helps bury bad behavior • Hacker News Board

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#311

Post by JohnPrine » Thu, Mar 09, 2023, 3:09 PM UTC

ElfinTrousers wrote: post #310

JamesBarney wrote: post #50

Earlier quoted context omitted.

There have been people for a long time who think the world is going to end. We can use your argument to dismiss anyone who fears any existential risk like asteroids, nuclear war, climate change, AI, etc... And since the world exists every group will be wrong except one. I think the risk of human extermination is much lower than many in this group believe because I think AI is much more compute bound than algorithm bo…

Asteroids, nuclear war, climate change, and AI. As the old song says "One of these things is not like the other". We know that asteroids exist and many large ones are uncomfortably close to our home. We know that nuclear weapons exist and have seen them used. We know that the climate is getting hotter (*). AI is...well, people losing their shit over ChatGPT (**) aside, AI is not going to be real enough to worry about…

Would you rather we wait until AIs are actually posing a threat for us to study ways to align them with human values? Tons of money already goes into fighting climate change and basically everyone on earth is aware of the threat it poses. The AI safety field is only about a decade old and is relatively unknown. Of course it makes sense to raise awareness there

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#312

Post by JohnPrine » Thu, Mar 09, 2023, 3:16 PM UTC

AnIdiotOnTheNet wrote: post #183

fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

There are many, but the biggest one is just that the possibility space is vast and unknowable and while drawing a line straight through it to AI doom is of course possible it doesn't say anything at all about likelihood, and it can't. It makes a ton of assumptions and expects everyone to just go along with them to reach its desired conclusion. AI certainly has risks, I don't think any reasonable human being doubts th…

> AI doom cult seems to think the worst outcomes are near certainties without really backing that up

What are you basing this on? There have been tons of arguments written about why these worst outcomes are likely. Read Bostrom's Superintelligence for example, or Yudkowsky's Intelligence Explosion Microeconomics.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#313

Post by ElfinTrousers » Thu, Mar 09, 2023, 3:42 PM UTC

JohnPrine wrote: post #311

ElfinTrousers wrote: post #310

Earlier quoted context omitted.

Asteroids, nuclear war, climate change, and AI. As the old song says "One of these things is not like the other". We know that asteroids exist and many large ones are uncomfortably close to our home. We know that nuclear weapons exist and have seen them used. We know that the climate is getting hotter (*). AI is...well, people losing their shit over ChatGPT (**) aside, AI is not going to be real enough to worry about…

Would you rather we wait until AIs are actually posing a threat for us to study ways to align them with human values? Tons of money already goes into fighting climate change and basically everyone on earth is aware of the threat it poses. The AI safety field is only about a decade old and is relatively unknown. Of course it makes sense to raise awareness there

> Would you rather we wait until AIs are actually posing a threat for us to study ways to align them with human values?

No, I would rather we wait until we are close enough to AI that we are talking about something concrete, rather than making wild speculations.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#314

Post by JohnPrine » Thu, Mar 09, 2023, 4:51 PM UTC

ElfinTrousers wrote: post #313

JohnPrine wrote: post #311

Earlier quoted context omitted.

Would you rather we wait until AIs are actually posing a threat for us to study ways to align them with human values? Tons of money already goes into fighting climate change and basically everyone on earth is aware of the threat it poses. The AI safety field is only about a decade old and is relatively unknown. Of course it makes sense to raise awareness there

> Would you rather we wait until AIs are actually posing a threat for us to study ways to align them with human values? No, I would rather we wait until we are close enough to AI that we are talking about something concrete, rather than making wild speculations.

Interesting. In your opinion, how should we decide when we're close enough that we're talking about something concrete?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#315

Post by mickelsen » Thu, Mar 09, 2023, 4:59 PM UTC

stubybubs wrote: post #22

Nexialist wrote: post #8

If you'd like to read more about the roots of Effective Altruism, ranging from old mailing lists, LessWrong, bitcoin, cults, AI and the surprising connections to many well known figures in tech, I highly recommend the 7-part Extropia's Childen series from Jon Evans [1] 1. https://aiascendant.substack.com/p/extropias-children-chapte...

Golly. I'm on part 2 and it sounds like they're about to invent Scientology. > The full post goes into more depth and nuance, but it's worth stressing how heavily Leverage relied on “debugging,” based on Anders's “Connection Theory.” Debugging consisted of — to paraphrase and oversimplify — opening up one's psyche, history, self, and emotional core to maximum vulnerability, generally to one's hierarchical superior, a…

This is all cult-like behavior, reminds me of the much documented Lyndon Larouche 'creative mentation' stuff, and pretty much any cult in their stage of grilling people to take down their preconceived notions. Now with this info, that's all I need to know about these characters.

Edit: found this https://www.reddit.com/r/SneerClub/

As someone said over there, it's about time to start putting lithium in the Bay Area water.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#316

Post by ElfinTrousers » Thu, Mar 09, 2023, 5:00 PM UTC

JohnPrine wrote: post #314

ElfinTrousers wrote: post #313

Earlier quoted context omitted.

> Would you rather we wait until AIs are actually posing a threat for us to study ways to align them with human values? No, I would rather we wait until we are close enough to AI that we are talking about something concrete, rather than making wild speculations.

Interesting. In your opinion, how should we decide when we're close enough that we're talking about something concrete?

Please see second asterisked point above.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#317

Post by JohnPrine » Thu, Mar 09, 2023, 5:14 PM UTC

ElfinTrousers wrote: post #316

JohnPrine wrote: post #314

Earlier quoted context omitted.

Interesting. In your opinion, how should we decide when we're close enough that we're talking about something concrete?

Please see second asterisked point above.

You seem to agree that if at some point in the next few decades AI will be something we need to worry about, so I'm trying to figure out exactly what it is you oppose.

Would you have opposed research into renewable energy in the 1970s since global warming was still a few decades away from being something we needed to worry about?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#318

Post by mitthrowaway2 » Thu, Mar 09, 2023, 6:54 PM UTC

simple-thoughts wrote: post #303

mitthrowaway2 wrote: post #293

Earlier quoted context omitted.

Between "I go to the grocery store because I've been socialized and trained that going to the grocery store regularly is a Good Thing, and I have adopted it as a terminal goal to which I know I should dedicate efforts", vs "I go to the grocery store because I'm out of carrots, my dinner recipe calls for carrots, and I think I can get some there", the latter model is not the one that strikes me as being stretched to e…

You’re arguing again from a position that assumes that entities have clearly defined terminal versus instrumental goals - which is precisely the position I reject. For instance in your example of “terminal goal is groceries” versus “terminal goal is hunger” neither of these describe how actual humans make decisions. Instead there’s a process, part biological and part environmental. The human checks the fridge then th…

Yeah, that's because re-evaluating your entire life plan and belief structure every second is expensive, and heuristics are cheap. People certainly have flaws in their thinking, which is why we fall prey to pyramid schemes, gambling, responding to pointless comment chains on HN, and so on. I don't disagree with this, and again, it's why we publish so many self-help books. But I believe it's our weakness and stupidity, not our superior intelligence and clear thinking, that traps us in bad habits.

So remind me of your original point? I believe you said it's "obviously wrong and laughable" that "an intelligent being can pursue stupid goals". Now here you are trying to convince me that humans are the ones who, like Pavlov's dogs, "pursue habits, processes, and heuristics that guide them through daily life even when those do not lead to any specific goal". Even when those habits involve repeatedly re-opening a fridge that you already know has no carrots in it, or salivating at a bell when you already know no food is coming.

So I'm confused how that proves your point about AGI. If I accept your view, it seems that if an AGI does merely no better than a human on this metric, I should anticipate all sorts of strange and irrational behaviour, including the pursuit of goals that would appear stupid, such as addiction to a reward channel. That does not seem to undermine the orthogonality thesis.

And the smarter the AGI gets, presumable the less it should lean on Pavlovian heuristics and the more it should make use of clear thought, which puts it more in my camp.

So that would apparently put the lower bound at "the AGI takes unexpected and irrational actions because it's not a rational agent and doesn't think coherently", and the upper bound at "the AGI takes unexpected and dangerous actions as rational steps toward an unaligned terminal goal".

I'm not sure where in this chain of thought it becomes laughably obvious that intelligence and goals are correlated, such that an AGI's increasing intelligence will tend it toward actions that we humans approve of, because anything else would be a "stupid goal"?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#319

Post by Vecr » Thu, Mar 09, 2023, 8:21 PM UTC

js8 wrote: post #301

Vecr wrote: post #271

Earlier quoted context omitted.

Right, but one of my points above is that AIs would quite possibly miss things or fail in general to make the comprehensive and sophisticated tradeoffs between efficiency and fragility on one side and inefficiency and robustness on the other. They could also miss cases where being less efficient is more fragile and being more efficient is less fragile. I really don't think leaving everything to AIs is a good idea, an…

Ever heard of reinforcement learning, AlphaGo, AlphaStar? AIs can already make that trade-off and rather well. AlphaGo was actually better than humans to play robust moves that cement its victory than riskier moves that increase the winning margin.

Right, but then you go back to the whole "AI alignment" thing and you need to make sure you reinforce the right things during training.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#320

Post by [deleted] » Fri, Mar 10, 2023, 1:43 PM UTC

JohnPrine wrote: post #317

ElfinTrousers wrote: post #316

Earlier quoted context omitted.

Please see second asterisked point above.

You seem to agree that if at some point in the next few decades AI will be something we need to worry about, so I'm trying to figure out exactly what it is you oppose. Would you have opposed research into renewable energy in the 1970s since global warming was still a few decades away from being something we needed to worry about?

[deleted]