Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

301–310 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#301

Post by js8 »

Vecr wrote: post #271
js8 wrote: post #142

Earlier quoted context omitted.

> most people's motivations are only loosely coupled with the goal (if there really is one) of making the world a more efficient place That's my point! Your personal AI (in collaboration with other AIs) can take care of figuring that out as needed. You don't need capitalism anymore.

Right, but one of my points above is that AIs would quite possibly miss things or fail in general to make the comprehensive and sophisticated tradeoffs between efficiency and fragility on one side and inefficiency and robustness on the other. They could also miss cases where being less efficient is more fragile and being more efficient is less fragile. I really don't think leaving everything to AIs is a good idea, an…

Ever heard of reinforcement learning, AlphaGo, AlphaStar? AIs can already make that trade-off and rather well. AlphaGo was actually better than humans to play robust moves that cement its victory than riskier moves that increase the winning margin.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#303

Post by simple-thoughts »

mitthrowaway2 wrote: post #293
simple-thoughts wrote: post #292

Earlier quoted context omitted.

Yes, you can stretch your model to try to explain why humans go to work. In reality, people do not need to go to work to “not starve to death” as you say. There are a myriad of ways to survive without working a daily job. Humans have to be socialized and trained to work a 9 to 5 job - there’s an entire education system structured to help create humans who view that as an acceptable goal. No what you are saying may no…

Between "I go to the grocery store because I've been socialized and trained that going to the grocery store regularly is a Good Thing, and I have adopted it as a terminal goal to which I know I should dedicate efforts", vs "I go to the grocery store because I'm out of carrots, my dinner recipe calls for carrots, and I think I can get some there", the latter model is not the one that strikes me as being stretched to e…

You’re arguing again from a position that assumes that entities have clearly defined terminal versus instrumental goals - which is precisely the position I reject. For instance in your example of “terminal goal is groceries” versus “terminal goal is hunger” neither of these describe how actual humans make decisions. Instead there’s a process, part biological and part environmental. The human checks the fridge then thinks “oh I feel hungry”. Is hunger the terminal goal or the fridge? That question doesn’t even make sense - it’s an interaction between the agent and the environment. Do Pavlov’s dogs have a terminal goal of “salivating to bells” or “salivating to food”? Again the question doesn’t make sense- the agent has built a habit from within a certain environment and the salivating is not goal directed. That’s why training works for dogs, and putting the fridge out of site reduces hunger in humans.

Think more carefully about the implications of multiple ways to survive here. Why do people pick one over the other? In a terminal/instrumental goal model, agents would pick the instrumental route that maximizes the return on the terminal goal. In reality we see that instead humans adopt habits, processes, and heuristics that guide them through daily life even when those do not lead to any specific goal.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#304

Post by fidgewidge »

ShredKazoo wrote: post #270
whatshisface wrote: post #265

Earlier quoted context omitted.

That blog post makes some giant unsupported assumptions, like that the reason for the failure of a few activists to stop gain-of-function research was that the government is fundamentally bad at addressing risks, and not that biologists might know more about the risk/benefit tradeoff than amateurs.

Was there even a serious policy discussion of banning gain-of-function research? Gain of function research is supposed to help with pandemics, right? Did it help with COVID?

GoF research was banned in the USA for long periods, that's why Fauci was outsourcing it to China and why DARPA refused to find that project.

No, not only has GoF not helped with COVID and probably caused it, but that's also true of the entire field of virology.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#305

Post by fidgewidge »

arisAlexis wrote: post #119
AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

There are top level scientistis warning of this apocalypse. Sam Altman which was the CEO of the very site you are posting and the CEO of OpenAI has warned starkly that there is a probability that humanity will go extinct from AI in his latest interview. Have you seen it? You think he is somehow bluffing or is he part of the same cult?

Sam Altman isn't a scientist but even if he was, so what, scientists have a reputation for constantly claiming the world will end unless they get more grant funding. Fear is a great motivator and makes people feel important.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#306

Post by fidgewidge »

scarmig wrote: post #231
mb7733 wrote: post #201

Earlier quoted context omitted.

> Corporations, when first developed, took out large chunks of Asia before anyone could stop them. What do you mean by this?

British East India Company, probably? At some point it had more soldiers and controlled a greater subject population than the government of Britain itself. I'm not sure it was ever consistently profitable , though, which is the main paperclip corporations ostensibly maximize. Which itself speaks to the problem: superficial objective functions don't always determine outcomes or behavior.

Beic was only a corporation in the most technical sense of the word. It became so powerful because the Queen had given it a government enforced monopoly from the start and the government kept extending that monopoly in return for loans and shares of the profits. The best modern equivalent would be Saudi Aramco, which also has its own "army", but it's meaningless to compare these sorts of organisations to modern uses of the word consideration. They're much closer to government departments.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#307

Post by LatteLazy »

PartiallyTyped wrote: post #273
LatteLazy wrote: post #238

Earlier quoted context omitted.

So far we have seen no power at all from ML. So spending large amounts of money making it "safe" (not that anyone knows how or even what it would look like) seems like an over reaction. With no power, comes no responsibility. But of course, if ML is a person's pet interest, they will likely think we should increase spending on it. That doesn't make that a smart decision...

I believe that you conflate “power” with capabilities wrt AGI. I’d argue that power is in relation to reach and the capacity to influence. In this regard, ML has quite a bit of power these days, from deciding whether you are allowed to take a loan, to driving engagement in social media, to predicting and influencing what you will watch, to the way our language is even used. There are also quite a few implicit ways th…

We can argue back and forth about the definition of power. Right now, ML has had very little actual impact. And it is very far from the sort of "skynet" general intelligence that people are afraid of. Those are just facts.

Personally I view it as no different to the hype over crypto or the fact we get a huge breakthrough in fusion every 3months but somehow never get any closer to an actual commercial reactor...

Meanwhile, we KNOW as a fact that climate change is going to be devastating. And there are numerous areas with nuclear wars brewing from far east to Europe.

I get that a super AI is a more interesting thing to worry about. But we need to deal with out actual, serious, issues first IMHO.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#308

Post by PartiallyTyped »

LatteLazy wrote: post #307
PartiallyTyped wrote: post #273

Earlier quoted context omitted.

I believe that you conflate “power” with capabilities wrt AGI. I’d argue that power is in relation to reach and the capacity to influence. In this regard, ML has quite a bit of power these days, from deciding whether you are allowed to take a loan, to driving engagement in social media, to predicting and influencing what you will watch, to the way our language is even used. There are also quite a few implicit ways th…

We can argue back and forth about the definition of power. Right now, ML has had very little actual impact. And it is very far from the sort of "skynet" general intelligence that people are afraid of. Those are just facts. Personally I view it as no different to the hype over crypto or the fact we get a huge breakthrough in fusion every 3months but somehow never get any closer to an actual commercial reactor... Meanw…

Given that ML has been exploited to drive polarization which fuels said wars and causes instability, can you claim that there is no impact with respect to said wars?

Power is not just about being more intelligent than us. An isolated airgaped super intelligence is powerless.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#309

Post by rafabulsing »

simple-thoughts wrote: post #204
rafabulsing wrote: post #177

Earlier quoted context omitted.

No goals are intrinsically stupid. The only reason that you might think some particular goal is stupid is that it goes against your goals. You could conceive of a super intelligent AI that came into existence with the goal of terminating itself. That would be an "stupid goal" from our perspective, since we have the goal of self-preservation really ingrained in our brains. But for a being that self-termination is the…

I’m not sure what you mean by “stupid” but it’s definitely not the colloquial meaning. A human seeking suicide is not stupid. In fact humans often do seek suicide because humans are playing social games where the maximizing move is to commit suicide. Supporting these people requires changing their context so that they have a wider and better variety of options available. This is just a specific example of why I rejec…

What do you mean by stupid? I used that word only because you used it first.

And I agree that humans do not only chase "breath feed sex", as I explicitly said that in my comment. We have other goals as well. But those are very much still there.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#310

Post by ElfinTrousers »

JamesBarney wrote: post #50
AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

There have been people for a long time who think the world is going to end. We can use your argument to dismiss anyone who fears any existential risk like asteroids, nuclear war, climate change, AI, etc... And since the world exists every group will be wrong except one. I think the risk of human extermination is much lower than many in this group believe because I think AI is much more compute bound than algorithm bo…

Asteroids, nuclear war, climate change, and AI. As the old song says "One of these things is not like the other". We know that asteroids exist and many large ones are uncomfortably close to our home. We know that nuclear weapons exist and have seen them used. We know that the climate is getting hotter (*).

AI is...well, people losing their shit over ChatGPT (**) aside, AI is not going to be real enough to worry about for a few more decades at least.

(*) Anyone who's about to regurgitate some fossil-fuel industry talking points in response, just save your breath.

(**) Above note goes double for this.

Post reply on HN