Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Universal Paperclips

decisionproblem.com

Post reply on HN
Print view View original

111–120 of 174 posts

Re: Universal Paperclips

#111

Post by spicybright »

majormajor wrote: post #109
madrox wrote: post #14

The paperclip maximizer is a thought experiment described by Swedish philosopher Nick Bostrom in 2003. It illustrates the existential risk that an artificial general intelligence may pose to human beings when it is programmed to pursue even seemingly harmless goals and the necessity of incorporating machine ethics into artificial intelligence design. The scenario describes an advanced artificial intelligence tasked w…

Some of these thought experiments seem very disconnected from how industry works. Like, we're saying "make as many paperclips as possible" as our instruction to this agent, not even "make as many as profitable" or "make up to X per day at a cost of less than Y per day"? The solution is proposed to be "program the AI to value human life" instead of the far simpler "put basic constraints on the process like you would i…

Even if we make an AI that wants to turn all matter into paper clips, we're so far away from an agent doing that I'm really not too worried.

I don't think there's any industry on earth that doesn't need humans in the loop somehow. Whether is mining raw material from the ground, loading stuff in machines for processing, and most importantly fixing broken down machines, robots are really bad at these things for the foreseeable future.

Not to mention AI needs constant electricity, which is really complicated and requires humans fixing a lot of stuff.

Re: Universal Paperclips

#112

Post by circuit10 »

spicybright wrote: post #111
majormajor wrote: post #109

Earlier quoted context omitted.

Some of these thought experiments seem very disconnected from how industry works. Like, we're saying "make as many paperclips as possible" as our instruction to this agent, not even "make as many as profitable" or "make up to X per day at a cost of less than Y per day"? The solution is proposed to be "program the AI to value human life" instead of the far simpler "put basic constraints on the process like you would i…

Even if we make an AI that wants to turn all matter into paper clips, we're so far away from an agent doing that I'm really not too worried. I don't think there's any industry on earth that doesn't need humans in the loop somehow. Whether is mining raw material from the ground, loading stuff in machines for processing, and most importantly fixing broken down machines, robots are really bad at these things for the for…

The thought experiment is about a superintelligence, which either wouldn’t need humans and could build some kind of robots or something even more effective that we haven’t thought of, or manipulate us into doing exactly what it “wants”

Also it’s a simplified example, it wouldn’t literally be paperclips but some other arbitrary goal (it shows how most goals takes to their absolute extreme won’t be compatible with human existence, even something that sounds harmless like making paperclips)

Re: Universal Paperclips

#113

Post by shhsshs »

burnished wrote: post #90
mjamil wrote: post #60

I don't know if people realize this, but the game now allows you to continue past the "you've converted all matter in the universe" point. You get to convert matter in other universes! (There is still a finite endpoint; I think there's only ~19 other universes you get to do this to.)

Does the gameplay loop change at that point?

It does not change in any substantial way, but you do get a boost to various stats in subsequent universes. You can also get "artifacts" by navigating the different universes which give some other significant stat boosts.

Re: Universal Paperclips

#114

Post by killerstorm »

d1str0 wrote: post #38
dylan604 wrote: post #33

Earlier quoted context omitted.

I like how the sci-fi authors have spent time thinking about what an advanced AI could do, yet those building the AI have not taken a moment's pause to consider what they are doing

Do you honestly believe that the only people who have thought about the ramifications of AI are Sci Fi authors? I can guarantee people who spend years researching and building advanced language models have thought about the ramifications of their work. This isn’t Jurassic Park.

That's not how it works. People publish papers demonstrating improvements without thinking about "ramifications".

Re: Universal Paperclips

#115

Post by slg »

majormajor wrote: post #109
madrox wrote: post #14

The paperclip maximizer is a thought experiment described by Swedish philosopher Nick Bostrom in 2003. It illustrates the existential risk that an artificial general intelligence may pose to human beings when it is programmed to pursue even seemingly harmless goals and the necessity of incorporating machine ethics into artificial intelligence design. The scenario describes an advanced artificial intelligence tasked w…

Some of these thought experiments seem very disconnected from how industry works. Like, we're saying "make as many paperclips as possible" as our instruction to this agent, not even "make as many as profitable" or "make up to X per day at a cost of less than Y per day"? The solution is proposed to be "program the AI to value human life" instead of the far simpler "put basic constraints on the process like you would i…

What about the engagement maximizing algorithms of the last decade plus which have seemingly helped fracture mature democracies by increasing extremism and polarization? Seems like we already have examples of companies using AI (or more specifically machine learning) to maximize some arbitrary goal without consideration for the real human harm that is created as a byproduct.

Re: Universal Paperclips

#116

Post by majormajor »

circuit10 wrote: post #110
majormajor wrote: post #109

Earlier quoted context omitted.

Some of these thought experiments seem very disconnected from how industry works. Like, we're saying "make as many paperclips as possible" as our instruction to this agent, not even "make as many as profitable" or "make up to X per day at a cost of less than Y per day"? The solution is proposed to be "program the AI to value human life" instead of the far simpler "put basic constraints on the process like you would i…

The point is it’s virtually impossible to put constraints on it that make it do what you want because if it’s more intelligent than you it can always think of something that you won’t, that’s technically within the rules you set but not at all intended. That’s why we’d need to make it care about the underlying intentions and values, but that’s also really hard

That seems like an attempt to set up a futile exercise in needle-threading that relies on narrow worst-case-scenario definitions of "superintelligence"/AGI.

It's too intelligent to restrict our constraints.

but

It's not too intelligent to be "aligned" to underlying intentions and values?

That approach doesn't even work on humans, why would it work on a superintelligence?

Re: Universal Paperclips

#118

Post by pbhjpbhj »

adamrezich wrote: post #44
jprete wrote: post #36

Earlier quoted context omitted.

One of the best ways to expand an incremental game’s gameplay is to gradually automate one layer, while introducing new mechanics in a higher layer. So one’s focus for optimization gradually shifts from older and better-understood mechanics to a higher level, where the gameplay is to manipulate the lower layer’s mechanics to make it run as quickly as a basic action did at the start.

a couple years ago a friend of mine got into botting Diablo II, which seemed like a whole game in itself, which made me think, how neat would that be, a game where you start off doing regular gameplay stuff, and then you eventually automate/optimize that gameplay at increasingly more meta levels of play. still think there's something to that idea...

ComputerCraft/EduCraft which was a Minecraft mod with Lua programmable robots that you could craft and set to work for you, gradually. Sounds similar to what you're going for here.

I wonder what other examples there are?

Re: Universal Paperclips

#120

Post by pbhjpbhj »

tleb_ wrote: post #51
adamrezich wrote: post #44

Earlier quoted context omitted.

a couple years ago a friend of mine got into botting Diablo II, which seemed like a whole game in itself, which made me think, how neat would that be, a game where you start off doing regular gameplay stuff, and then you eventually automate/optimize that gameplay at increasingly more meta levels of play. still think there's something to that idea...

That sounds like the concept of Factorio: the abstraction level we play at rises with time. Most of the achievements are about skipping manual labor at various points in time.

Did Factorio have blueprints? That seemed like one of the greatest advances in Dyson Sphere Program, when you could lay down a whole factory unit at once; emphasised the move to greater levels of abstraction.

Mind you, I'd have liked a lite version that reduced the tech tree by maybe a third but still enabled you to get all the advancements without devoting your whole life to it ;o)

Post reply on HN