Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

291–300 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#291

Post by mitthrowaway2 »

simple-thoughts wrote: post #287
mitthrowaway2 wrote: post #257

Earlier quoted context omitted.

> The orthogonality thesis is an unproven assumption A safety mindset would suggest that rather than disregarding it until proven true, we should worry about it until proven false. > meaning an intelligent being can pursue stupid goals. It would pursue very intelligent instrumental goals , but the terminal goal is a free variable and I don't think there exists any measure by which terminal goals can be considered sma…

This “safety mindset” is the same as Pascal’s wager and has the same flaws. There’s no clear delineation in real entities between instrumental and terminal goals. Yeah if you’re talking to a brainwashed religious fanatic, definitely it can seem counterintuitive to the doctrines they believe.

> There’s no clear delineation in real entities between instrumental and terminal goals.

In healthy individuals, yes there absolutely is. Terminal goals are the ones you pursue for their own sake; instrumental goals are the ones you pursue as part of a plan to pursue a terminal goal, or another instrumental goal which connects to a terminal goal. Most people go to work in the morning not as a terminal goal, but as an instrumental goal; employment is in service of another instrumental goal of earning money; earning money is in service of a terminal goal of not starving to death. This is not exactly controversial stuff here. Some people do get so focused on an instrumental goal like "earning money" that they develop tunnel-vision and forget what terminal goal that money was originally in service of, but that's something most of them will eventually realize and then write a self-help book about.

Anyway, it takes intelligence to decide what your instrumental goals should be, such as whether there's perhaps a cleverer way to make money than by going to work for your boss each morning, but there's no way in which intelligence will help you choose your terminal goals. For the most part they aren't something you can even consciously choose.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#292

Post by simple-thoughts »

mitthrowaway2 wrote: post #291
simple-thoughts wrote: post #287

Earlier quoted context omitted.

This “safety mindset” is the same as Pascal’s wager and has the same flaws. There’s no clear delineation in real entities between instrumental and terminal goals. Yeah if you’re talking to a brainwashed religious fanatic, definitely it can seem counterintuitive to the doctrines they believe.

> There’s no clear delineation in real entities between instrumental and terminal goals. In healthy individuals, yes there absolutely is. Terminal goals are the ones you pursue for their own sake; instrumental goals are the ones you pursue as part of a plan to pursue a terminal goal, or another instrumental goal which connects to a terminal goal. Most people go to work in the morning not as a terminal goal, but as an…

Yes, you can stretch your model to try to explain why humans go to work.

In reality, people do not need to go to work to “not starve to death” as you say. There are a myriad of ways to survive without working a daily job.

Humans have to be socialized and trained to work a 9 to 5 job - there’s an entire education system structured to help create humans who view that as an acceptable goal.

No what you are saying may not be controversial in your little community but the AI panic is mostly isolated to a small community in a small corner of the USA.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#293

Post by mitthrowaway2 »

simple-thoughts wrote: post #292
mitthrowaway2 wrote: post #291

Earlier quoted context omitted.

> There’s no clear delineation in real entities between instrumental and terminal goals. In healthy individuals, yes there absolutely is. Terminal goals are the ones you pursue for their own sake; instrumental goals are the ones you pursue as part of a plan to pursue a terminal goal, or another instrumental goal which connects to a terminal goal. Most people go to work in the morning not as a terminal goal, but as an…

Yes, you can stretch your model to try to explain why humans go to work. In reality, people do not need to go to work to “not starve to death” as you say. There are a myriad of ways to survive without working a daily job. Humans have to be socialized and trained to work a 9 to 5 job - there’s an entire education system structured to help create humans who view that as an acceptable goal. No what you are saying may no…

Between "I go to the grocery store because I've been socialized and trained that going to the grocery store regularly is a Good Thing, and I have adopted it as a terminal goal to which I know I should dedicate efforts", vs "I go to the grocery store because I'm out of carrots, my dinner recipe calls for carrots, and I think I can get some there", the latter model is not the one that strikes me as being stretched to explain human behaviour.

As for there being "a myriad of ways to survive without working a daily job", congratulations! Your intelligence has allowed you to identify alternative instrumental goals that provide a path to your terminal goal; now you can rank them and choose the best option. You can also grow carrots in the garden or ask your neighbour if they have any, or ask your spouse to pick some up on the way home. Your intelligence will do the work and find a way. But your intelligence isn't what will guide you toward preferring carrot soup over parsnip soup, and preferring parsnip soup over fasting.

I'm not American and not in the USA, btw.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#294

Post by Vecr »

HDThoreaun wrote: post #280
Vecr wrote: post #267

Earlier quoted context omitted.

Yes, spoilers though, but I don't actually think that explains it. Are there any indications in cannon that Tom Riddle was ever interested in polyamory? I'm pretty sure voldemort is not interested in other people at all after his soul gets all divided, but I don't know about before that. Riddle is not from the US west coast either, so it's kind of dubious.

I don't think the story is trying to take the characters from cannon, but take the world and completely reimagine the characters. The original characters just serve as foils for their reimagined versions. I never really understood complaints about fanfic authors not sticking to cannon character traits. Isn't the point to create new stories that are loosely linked to the original? Anyway, HPMOR is more pop philosophy…

I guess so. As I said before it's essentially a morality story, but in that sense I think it's evidence for the author having a bias toward polyamory, because there's very little or no in-universe reasons, and there's no broad social acceptance problem in not having polyamory mentioned at all. As an example, I think there would be social acceptance problems if the story had no women or clothing in it. Polyamory is no where near that level.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#295

Post by fluxinflex »

CatWChainsaw wrote: post #249
fluxinflex wrote: post #131

I doubt very much that the poorest people on this planet care much about AIs. People living in absolute poverty in Africa, India, China etc. I also doubt they would be - negatively - affected if a rogue AI would turn out the lights in the electronicified world. Places where AIs aren't used to command peoples lives will probably not even realise that an AI apocalypse had happened. Perhaps a case of western navel-gazin…

By that logic, uncontacted tribes shouldn't be negatively affected by the modern world, and the Sentinelese don't need to concern themselves with sea level rise even if their island is completely swallowed.

Not all uncontacted tribes live on the coast. But yes, there are very few tribes who haven't been influenced by the "modern" world.

I always wonder why these tribes haven't yet invented the iPhone? Obviously since they don't seem to need it.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#296

Post by elmomle »

operatingthetan wrote: post #18
elmomle wrote: post #17

Earlier quoted context omitted.

Thanks for sharing that reference. I started out being intrigued, but he totally lost me with some pretty unsubstantiated absolutist statements, e.g. "anyone smart and independent-minded will have views on race and sex well outside the liberal mainstream" and "men are sure to be a disproportionate share of leaders for biological reasons". The first statement is categorically false or otherwise implies the author's en…

The photo of Aella in a bra with the caption "The rare rationalist female" wasn't a red flag? More seriously, yes that piece is dripping with sexism. >Women will never make up a significant portion of self-made billionaires or outstanding philosophers, but they can get a majority of BAs. Academia practices affirmative action in everything from admissions to journal submissions and citations practices, artificially re…

Yeah, that was also a massive red flag once I got to it, but I wanted to give a more considered critique.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#297

Post by CatWChainsaw »

fluxinflex wrote: post #295
CatWChainsaw wrote: post #249

Earlier quoted context omitted.

By that logic, uncontacted tribes shouldn't be negatively affected by the modern world, and the Sentinelese don't need to concern themselves with sea level rise even if their island is completely swallowed.

Not all uncontacted tribes live on the coast. But yes, there are very few tribes who haven't been influenced by the "modern" world. I always wonder why these tribes haven't yet invented the iPhone? Obviously since they don't seem to need it.

In order to make an iPhone there was the basis of cell phones, and telephony and radio in general. And the mining of materials that such tribes may not be in proximity to or be able to mine at such intensities given their low populations and tech levels.

For some reason you seem to expect a civilization with the technological sophistication of globalized society with 10,000+ years and billions of people and countless global resources fueling them to suddenly emerge in the small geographies and populations of uncontacted tribes? If you were trying to be witty, you came up short.

Edit: or were you trying to play off "negatively affected by the modern world" as, "well they don't have iPhones so they don't have the Social Dilemma, everything's peachy" instead of the blatantly obvious "we destroyed their world and they had no clue" angle...

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#298

Post by skybrian »

d0mine wrote: post #276
skybrian wrote: post #21

In case you're thinking you saw this before, there was similar article a month ago in Time: https://news.ycombinator.com/item?id=34646361

It looks like a hit piece. Someone is against rationality regarding the question of AI safety (any real practical consequences is to censor AI under _pretense_ of AI safety, Perhaps, the articles are the preventive strikes to discredit the AI safety experts that might point out the obvious).

That's a rather specific accusation to make without evidence. Possibly, a journalist might have other reasons to cover a story? Not necessarily good ones, but still...

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#299

Post by pksebben »

arisAlexis wrote: post #285
pksebben wrote: post #261

Earlier quoted context omitted.

it's a little annoying that the "AI Impacts" paper seems to be three MIRIs in a trenchcoat, considering that this article is about EA's AI agenda and by extension Singularity/MIRI. There is a slight conflict of interest here - as the publishers of the paper are also taking funding specifically to "deal with the threat" they're publishing on.

Yes if we have scientists that are getting paid to research a certain risk it’s inevitable that they have conflict of interest. Otherwise how would it be done?

I'll admit, there isn't actually a good solution to this. It's one of the core failures of the way we organize society and its resources.

I find this kind of fascinating, really. Because of the way we sort work and create institutions, there's a structural hook that ties in the persistence of the problem (or the creation of a perception of the problem) with the longevity and motivation of the institution / entity that is attempting to solve it.

Yeah, you could contract out to a third party (which would be better), but you're still establishing a relationship where their future earnings are dependent on coming up with a certain result, unless everyone involved is incredibly self aware and conscientious and is willing to work to put themselves out of a job. Not impossible, but a rarity.

Take the DEA, for example. Every indicator points to deprecation as the way forward. The science says that treating drugs like a crime is ineffective and creates all sorts of externalities, and there isn't even broad public support for the War On Drugs. It's expensive, ineffective, dangerous, wasteful. And it keeps an absolute fuckton of people employed, at all levels. These people have a vested interest in maintaining the status quo, if for no other reason than their positions afford them power, quality of life, etc. So they lobby to keep stuff illegal and go on campaigns attempting to convince people that they'll peel themselves if they get high.

I'm not suggesting that EA and this AI thing is comparable to the DEA, mind you. If given the choice to keep one I'd be hanging out with you guys. That said, I'm ever leery of surveys in general, and there's a recursiveness to using your own data to back up your claims that doesn't sit well. If it's truly something that warrants money / time / attention, there's gotta be external sources that corroborate - Which I don't necessarily doubt, but that's not the thing we're looking at, here.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#300

Post by fasterik »

Sol- wrote: post #129
TheDong wrote: post #112

Earlier quoted context omitted.

EAs premise includes basically "all human lives are equal. Saving one stranger's life is just as valuable as saving your father's life (more valuable actually if the stranger has more years left than your dad), so you should donate your kidney to a stranger, not your father" "All human lives are equal" then expands to "and there are more unborn lives than there are currently living lives, so it's more important to sa…

How they value unborn future lives also veers into absurdity when you realize that the logical goal from a utilitarian perspective then is to populate the universe with trillions upon trillions of humans, just as long as each human at least has some infinitesimally small net happiness. You don't need them to be particularly happy, just happy enough to barely not want to kill themselves. I think I'd rather prioritize…

That's a logical consequence of one possible view in population ethics, which is orthogonal to utilitarianism.

There are equally counter-intuitive consequences of discounting future lives or not valuing them at all. What makes this specific point in time so much more important than every other point in time? Can we do anything we want to make our lives better regardless of future consequences?

Post reply on HN