Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Universal Paperclips

decisionproblem.com

Post reply on HN
Print view View original

131–140 of 174 posts

Re: Universal Paperclips

#131

Post by jrobbins »

adamrezich wrote: post #11

what're some other interesting "incremental games" out there these days? for me at least, just seeing numbers go up and get huge (Swarm Simulator) doesn't really do it for me. part of what makes Universal Paperclips so good is that, like Candy Box, a huge part of the joy is uncovering entirely new gameplay systems as you progress. A Dark Room was neat in that it brought the idea of a coherent narrative that you (some…

I made a cooperative team incremental game about software testing that has silly trivia questions. It won't waste your team's whole day because the story plays out in exactly 20 minutes.

https://greens-io.appspot.com

Re: Universal Paperclips

#132

Post by parpfish »

spicybright wrote: post #111
majormajor wrote: post #109

Earlier quoted context omitted.

Some of these thought experiments seem very disconnected from how industry works. Like, we're saying "make as many paperclips as possible" as our instruction to this agent, not even "make as many as profitable" or "make up to X per day at a cost of less than Y per day"? The solution is proposed to be "program the AI to value human life" instead of the far simpler "put basic constraints on the process like you would i…

Even if we make an AI that wants to turn all matter into paper clips, we're so far away from an agent doing that I'm really not too worried. I don't think there's any industry on earth that doesn't need humans in the loop somehow. Whether is mining raw material from the ground, loading stuff in machines for processing, and most importantly fixing broken down machines, robots are really bad at these things for the for…

i agree that there's no way to get humans out of the loop. somebody set up this machine to make paperclips because some human(s) wanted/needed paperclips. eventually, one of those people would realize "we have enough paperclips. let's turn off the paperclip making machine".

this nightmare scenario really only plays out if paperclip machine develops some sort of self-preservation instinct and has the means to defend/protect itself from being disabled. Building a machine capable of that seems a) like fantastical scifi and b) easily preventable.

Re: Universal Paperclips

#133

Post by sfink »

Sardtok wrote: post #103
cubefox wrote: post #83

Earlier quoted context omitted.

> It's also hard to understand why an AI smart enough to paperclip the world wouldn't also be smart enough to realize the futility in doing so. Terminal goals can't be futile, since they do not serve to achieve other (instrumental) goals. Compare: Humans like to have protected sex, watch movies, eat ice cream, even though these activities might be called "futile" or "useless" (by someone who doesn't have those goals)…

It's not like you want to eat ice cream constantly, even if it means making everything into ice cream. Of course the premise becomes that the AI has been instructed to make paperclips. They should have hired a better prompt engineer, capable of actually specifying the goals more clearly. I don't think an AI that eradicates humankind, will have such simplistic goals, if an AI ever becomes the end of humans. Cybermen,…

Yes, they should just write prompts without bugs. Can't be that much harder than writing software without bugs.

Re: Universal Paperclips

#134

Post by feoren »

dylan604 wrote: post #33
actionfromafar wrote: post #30

Earlier quoted context omitted.

PKD keeps surprising me.

I like how the sci-fi authors have spent time thinking about what an advanced AI could do, yet those building the AI have not taken a moment's pause to consider what they are doing

That's because the people who are building the AI actually know how it works, understand how fundamentally simple it all is, and know that there's no room for consciousness to magically emerge. The current state of AI is not so much a story of any kind of "intelligence" being amazing, but rather the sum total of humanity's data being amazing. The amazing feats LLMs perform come from the words we all wrote, not the code they wrote. The code just unlocks the previously latent power of all that data.

It is nothing close to being an actual intelligence, regardless of how much we anthropomorphize it. We also anthropomorphize stick figures, stuffed animals, and weighted companion cubes.

Re: Universal Paperclips

#135

Post by mikea1 »

parpfish wrote: post #130
mikea1 wrote: post #107

Earlier quoted context omitted.

Another explanation is that there are those who considered and thoughtfully weighed the ramifications, but came to a different conclusion. It is unfair to assume a decision process was agnostic to harm or plain ignorant. For example, perhaps the lesser-evil argument played a role in the decision process: would a world where deep fakes are ubiquitous and well-known by the public be better than a world where deep fakes…

there's also the issue that most of the AI catastrophizing is a pretty clear slipperyslope argument: if we build ai AND THEN we give it a stupid goal to optimize AND THEN we give it unlimited control over its environment, something bad will happen. the conclusion is always "building AI is wrong" and not "giving AI unrestricted control of critical systems is wrong"

I completely agree that's a valid argument. I just think it is rational for someone to come to a different conclusion, given identical priors.

Re: Universal Paperclips

#136

Post by circuit10 »

majormajor wrote: post #121
circuit10 wrote: post #112

Earlier quoted context omitted.

The thought experiment is about a superintelligence, which either wouldn’t need humans and could build some kind of robots or something even more effective that we haven’t thought of, or manipulate us into doing exactly what it “wants” Also it’s a simplified example, it wouldn’t literally be paperclips but some other arbitrary goal (it shows how most goals takes to their absolute extreme won’t be compatible with huma…

What about "most arbitrary goals are incompatible with human existence" requires super-human intelligence? A human who wanted to "build as many paperclips as possible" could cause a great deal of destruction today. A human who wanted to accumulate as much wealth as possible could, too. EDIT: maybe a better way of articulating my complaints about this famous thought experiment is that it's supposed to be making a poin…

> What about "most arbitrary goals are incompatible with human existence" requires super-human intelligence?

The "taken to the absolute extreme" part.

> A human who wanted to "build as many paperclips as possible" could cause a great deal of destruction today.

Maybe, but a) no one really wants that (at least not as their only desire above all else) and b) we aren't superintelligent so it's hard to gain enough control and power and plan well enough to do it that well

> talking about a goal that has sub-human-intelligence sophistication

There is no reason a simple goal can't be followed in an intelligent way or vice versa. This is called the "orthogonality thesis". There's a good video about it here: https://www.youtube.com/watch?v=hEUO6pjwFOo

Re: Universal Paperclips

#137

Post by circuit10 »

seventhtiger wrote: post #122
circuit10 wrote: post #110

Earlier quoted context omitted.

The point is it’s virtually impossible to put constraints on it that make it do what you want because if it’s more intelligent than you it can always think of something that you won’t, that’s technically within the rules you set but not at all intended. That’s why we’d need to make it care about the underlying intentions and values, but that’s also really hard

The basic premise is that it has somewhere in it that is telling it to make more paperclips. Put the constraints there. If you're saying such an AI would be too smart to be a simple paperclip maximizer, then I'd agree, but then what's the point of the thought experiment if a paperclip maximizer is impossible.

> The basic premise is that it has somewhere in it that is telling it to make more paperclips. Put the constraints there.

What constraints do you suggest? If it's just changing "make as many paperclips as possible" to "make at least x number of paperclips" (putting a cap on the reward it gets), here's a good explanation of why that doesn't really work: https://www.youtube.com/watch?v=Ao4jwLwT36M

If you're suggesting limiting the types of actions it can take, then to do that to the point that a superintelligence can't find a way around it (maybe letting it choose between one of two options and then shutting it down and never using it again) would make it not very useful, so you'd be better off just not making it at all

> If you're saying such an AI would be too smart to be a simple paperclip maximizer

No, that's not what I'm saying. Any goal is compatible with any level of intelligence, there is no reason why it wouldn't be possible to follow a simple goal in a complex way. Again here's a video about that: https://www.youtube.com/watch?v=hEUO6pjwFOo

Re: Universal Paperclips

#138

Post by goatlover »

ASalazarMX wrote: post #105
goatlover wrote: post #65

Earlier quoted context omitted.

Or the researchers don't think existential threats are realistic, and paper maximizing thought experiments are silly. Maybe they're wrong, but maybe not. It's easy to imagine AI takeover scenarios by giving them unlimited powers, it's hard to show the actual path to such abilities. It's also hard to understand why an AI smart enough to paperclip the world wouldn't also be smart enough to realize the futility in doing…

> AGIs would understand we don't want to paperclip the world. Even if they did, what if they aren't smart enough for eloquent humans to convince them it's for the greater good. True AGIs will need a moral code to match their intelligence, and someone will have to decide what's good and bad to make that moral code.

Then they won't be smart enough to paperclip the world. No human organization can do that.

Re: Universal Paperclips

#139

Post by bananapub »

feoren wrote: post #134
dylan604 wrote: post #33

Earlier quoted context omitted.

I like how the sci-fi authors have spent time thinking about what an advanced AI could do, yet those building the AI have not taken a moment's pause to consider what they are doing

That's because the people who are building the AI actually know how it works, understand how fundamentally simple it all is, and know that there's no room for consciousness to magically emerge. The current state of AI is not so much a story of any kind of "intelligence" being amazing, but rather the sum total of humanity's data being amazing. The amazing feats LLMs perform come from the words we all wrote, not the co…

that's indeed not a sensible worry, but the actual consequences on society of such things are extremely real and already happening, and are something the people involved seem either uninterested in worrying about or actively encouraging.

Re: Universal Paperclips

#140

Post by parpfish »

mikea1 wrote: post #135
parpfish wrote: post #130

Earlier quoted context omitted.

there's also the issue that most of the AI catastrophizing is a pretty clear slipperyslope argument: if we build ai AND THEN we give it a stupid goal to optimize AND THEN we give it unlimited control over its environment, something bad will happen. the conclusion is always "building AI is wrong" and not "giving AI unrestricted control of critical systems is wrong"

I completely agree that's a valid argument. I just think it is rational for someone to come to a different conclusion, given identical priors.

If it wasn’t clear, I agree with your parent comment
Post reply on HN