Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Universal Paperclips

decisionproblem.com

Post reply on HN
Print view View original

151–160 of 174 posts

Re: Universal Paperclips

#151

Post by Sohcahtoa82 »

pbhjpbhj wrote: post #120
tleb_ wrote: post #51

Earlier quoted context omitted.

That sounds like the concept of Factorio: the abstraction level we play at rises with time. Most of the achievements are about skipping manual labor at various points in time.

Did Factorio have blueprints? That seemed like one of the greatest advances in Dyson Sphere Program, when you could lay down a whole factory unit at once; emphasised the move to greater levels of abstraction. Mind you, I'd have liked a lite version that reduced the tech tree by maybe a third but still enabled you to get all the advancements without devoting your whole life to it ;o)

Factorio had blueprints wayyyyy before DSP.

Re: Universal Paperclips

#152

Post by seventhtiger »

circuit10 wrote: post #137
seventhtiger wrote: post #122

Earlier quoted context omitted.

The basic premise is that it has somewhere in it that is telling it to make more paperclips. Put the constraints there. If you're saying such an AI would be too smart to be a simple paperclip maximizer, then I'd agree, but then what's the point of the thought experiment if a paperclip maximizer is impossible.

> The basic premise is that it has somewhere in it that is telling it to make more paperclips. Put the constraints there. What constraints do you suggest? If it's just changing "make as many paperclips as possible" to "make at least x number of paperclips" (putting a cap on the reward it gets), here's a good explanation of why that doesn't really work: https://www.youtube.com/watch?v=Ao4jwLwT36M If you're suggesting…

The most intelligent person ever born could still die to a gun. In these discussions superintelligent AI can be more accurately described as "the genie" or "God". If you assume omniscience and omnipotence I guess nothing else matters. But intelligence is not equal to power, and never has.

Second, if you are able to set a goal then during this setting you can set many constraints, even fundamental ones. There is no reason the goal is more fundamental than the constraint. If I approve, make paperclips. Efficiently make 100 paperclips.

It's the duality of being able to set a rule but not being able to set a constraint that I find a strange concept. I lean towards the picture of not being able to set goals nor constraints at all.

Re: Universal Paperclips

#153

Post by circuit10 »

seventhtiger wrote: post #152
circuit10 wrote: post #137

Earlier quoted context omitted.

> The basic premise is that it has somewhere in it that is telling it to make more paperclips. Put the constraints there. What constraints do you suggest? If it's just changing "make as many paperclips as possible" to "make at least x number of paperclips" (putting a cap on the reward it gets), here's a good explanation of why that doesn't really work: https://www.youtube.com/watch?v=Ao4jwLwT36M If you're suggesting…

The most intelligent person ever born could still die to a gun. In these discussions superintelligent AI can be more accurately described as "the genie" or "God". If you assume omniscience and omnipotence I guess nothing else matters. But intelligence is not equal to power, and never has. Second, if you are able to set a goal then during this setting you can set many constraints, even fundamental ones. There is no re…

Intelligence definitely helps with gaining power. Humans aren’t very strong yet we have a lot of power thanks to our intelligence.

You can set constraints just fine. It’s simply a part of the goal: “do x without doing y”. It’s just really hard to find the right constraints, no simple one works.

For example “if I approve, make paperclips” - so it gets more reward if you approve? What’s to stop it from manipulating you into thinking nothing is wrong so you always approve? “Efficiently make 100 paperclips.” I already linked a video on why capping the reward like that doesn’t work, but if you don’t want to watch it the gist is that for your suggestion it may just make a maximiser which is pretty guaranteed to make at least 100, and is pretty efficient because it’s not doing much work itself. Then the maximiser kills us all

Re: Universal Paperclips

#154

Post by mikea1 »

parpfish wrote: post #140
mikea1 wrote: post #135

Earlier quoted context omitted.

I completely agree that's a valid argument. I just think it is rational for someone to come to a different conclusion, given identical priors.

If it wasn’t clear, I agree with your parent comment

My main takeaway from Bostrom's Superintelligence is that a super intelligent AI cannot be contained. So, the slippery slope argument, often derided as a bad form of logic, kind of holds up here.

Re: Universal Paperclips

#155

Post by actionfromafar »

olddustytrail wrote: post #144
parpfish wrote: post #130

Earlier quoted context omitted.

there's also the issue that most of the AI catastrophizing is a pretty clear slipperyslope argument: if we build ai AND THEN we give it a stupid goal to optimize AND THEN we give it unlimited control over its environment, something bad will happen. the conclusion is always "building AI is wrong" and not "giving AI unrestricted control of critical systems is wrong"

The massive flaw in your argument is your failure to define "we". Replace the word "we" with "a psychotic group of terrorists" in your post and see how it reads.

Or "board of directors beholden to share-holders".

Re: Universal Paperclips

#156

Post by actionfromafar »

Karunamon wrote: post #141
willcipriano wrote: post #75

Earlier quoted context omitted.

The people building it know it isn't AI, the people selling it call it that.

To say that something like GPT-4 does not count as "AI" requires a gargantuan shifting of goalposts from where they were at this time last year.

So it was ever since computers started playing chess.

Re: Universal Paperclips

#157

Post by parpfish »

olddustytrail wrote: post #144
parpfish wrote: post #130

Earlier quoted context omitted.

there's also the issue that most of the AI catastrophizing is a pretty clear slipperyslope argument: if we build ai AND THEN we give it a stupid goal to optimize AND THEN we give it unlimited control over its environment, something bad will happen. the conclusion is always "building AI is wrong" and not "giving AI unrestricted control of critical systems is wrong"

The massive flaw in your argument is your failure to define "we". Replace the word "we" with "a psychotic group of terrorists" in your post and see how it reads.

If you’re talking about some group of evildoers that deploy ai in a critical system to do evil… the issue is why do they have control to the critical system? Surely they could jump straight to their evil plot with the ai at all
Post reply on HN