Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dude, You Broke the Future (2017)

antipope.org

Post reply on HN
Print view View original

41–50 of 147 posts

Re: Dude, You Broke the Future (2017)

#41

Post by Nasrudith »

lukev wrote: post #34
jengleton wrote: post #26

Earlier quoted context omitted.

That's fair. I interpreted the comparison as being between "AI implemented as a property of human organizations" and "AI implemented as a powerful search algorithm". While corporations can certainly be dangerous, they're still made of people and thus are unlikely to want to do things like "convert the entire mass of the solar system into dollar bills" (and even if they did, they'd have a hard time doing it). A suffic…

How is "converting the entire mass of the solar system into dollar bills" conceptually different from "converting the habitability of Earth into dollar bills" or "converting the health problems of human beings into a maximal amount of dollar bills", which is precisely what many corporations _actually do_? The "corporation as AI" metaphor isn't about some abstract future possibility, it's an explanatory mechanism for…

That assumes an ideal corporation in the physics sense - real world ones have "corruption" in the form of their employees engaging in their own agenda. Take sexual harassment, passing up talent from bigotry, and office politics. They actively harm the profitability and yet they exist at all levels.

The incentives are fundamentally what shape the systems including the corporations. Blaming corporations alone is a simplification - the same incentives converge to the same outcomes akin to how power vacuums are filled by warlords.

Re: Dude, You Broke the Future (2017)

#42

Post by Chathamization »

jengleton wrote: post #33
Chathamization wrote: post #28

Earlier quoted context omitted.

> In the sense that a corporation's maximum intelligence is likely to be in the same ballpark as the smartest humans, it can't be seen as a true superintelligence. In what sense? Certainly the ability of large groups of people to research and create new technology exceed the ability of a single individual. The R&D capabilities of a large corporation are going to vastly exceed that of a lone hermit in the woods.

So I think the question is: when a group of people work together on something, can they generate ideas which are better than the best idea of any member of the group? If so, how much better? How does the effect scale as the size of the group increases? An R&D lab certainly can do more work than any single person could do in their lifetime, but are the ideas fundamentally better? Can you point to an idea and say: "no…

I'd say yes. Do you think the ideas that came from the Manhattan Project didn't exceed that of the best individual involved?

Or for a more extreme example - consider the totality of human civilization as an example of groups of humans working together. People like Einstein and Hawking wouldn't have been able to have had the ideas they did if they were dropped in 20,000 BC. The knowledge we have now isn't simply the work of individual animals with extraordinary thoughts, it's the result of large groups of these animals working together.

Now certainly the benefits in corporate R&D labs don't come anywhere close, but I find it hard to believe that they simply drop to zero.

Re: Dude, You Broke the Future (2017)

#43

Post by Nasrudith »

claudiawerner wrote: post #11
autokad wrote: post #10

>"Science fiction is written by people embedded within a society with expectations and political assumptions that bias us towards looking at the shiny surface of new technologies rather than asking how human beings will use them, and to taking narratives of progress at face value rather than asking what hidden agenda they serve." And be prepared for the most agenda ridden text you have read in a long time.

That's not necessarily a bad thing; in fact, most works (and specifically blogs) online have an "agenda".

Not having any agenda is a deeply unhealthy mental state as it implies a complete apathy to all outcomes. It is just that most "normal" agendas are effectively invisible as a Huffman encoded to a 0 bit default. Not wanting to be tortured to death is an agenda technically but contributes so little information that it only makes sense to state the opposite.

Re: Dude, You Broke the Future (2017)

#44

Post by pitmanyor »

jengleton wrote: post #33
Chathamization wrote: post #28

Earlier quoted context omitted.

> In the sense that a corporation's maximum intelligence is likely to be in the same ballpark as the smartest humans, it can't be seen as a true superintelligence. In what sense? Certainly the ability of large groups of people to research and create new technology exceed the ability of a single individual. The R&D capabilities of a large corporation are going to vastly exceed that of a lone hermit in the woods.

So I think the question is: when a group of people work together on something, can they generate ideas which are better than the best idea of any member of the group? If so, how much better? How does the effect scale as the size of the group increases? An R&D lab certainly can do more work than any single person could do in their lifetime, but are the ideas fundamentally better? Can you point to an idea and say: "no…

R&D labs aren't necessarily restricted to a single research group with a unique, well defined goal. Depending on their organization they can easily integrate more or less isolated units of self- or loosely-directed research driven by single individuals or small teams. Properly managed, I can see this kind of heterogeneity of agents performing at a level no single human ever could. There's a analogy with boosting to be made somewhere but I can't care to flesh it out right now.

Re: Dude, You Broke the Future (2017)

#45

Post by pdonis »

einpoklum wrote: post #27
jariel wrote: post #22

Corporations are nothing but a group of actors working together to achieve some outcome, part of which may be profit, and it's absurd to think that they are a 'Modern Era', or even Western concept as every single land-owning farmer, artisan, banker, merchant in history was de-facto a form of 'corporation'. Moreover, many institutions with origins in antiquity (schools, governments, NGO's) are close enough to the orga…

Corporations are literally the something-extra over a "group of actors working together".

No, they aren't. Corporations don't magically do things that the group of actors working together don't do.

Re: Dude, You Broke the Future (2017)

#46

Post by worldsayshi »

ALittleLight wrote: post #19

I found a couple errors early on that dissuaded me from continuing to read. One was the idea that the author could rule out the singularity because he wasn't aware of progress toward self-motivated AI that would be something like our own intelligence. This seems like a limited view to me because we wouldn't need self-motivation at all in AI to hit the singularity. Humans can supply the self motivation. Suppose I'm us…

> Tesla [...] is clearly not a different class of intelligence from [...] corporations

I believe he just used Tesla as an example of corporation not as something new or special.

> the point of the paperclip thought experiment is, broadly, that an AI's interests might not be aligned with our own and that misalignment may cause serious problems.

But (some) corporations clearly do share this aspect to some extent. They maximize profit while environmental concerns are not prioritized.

And he pointed out later that corporations are slightly more complex than just maximizing 'one' thing. But a range of things and most importantly profit.

Re: Dude, You Broke the Future (2017)

#47

Post by zaphar »

jengleton wrote: post #40
lukev wrote: post #34

Earlier quoted context omitted.

How is "converting the entire mass of the solar system into dollar bills" conceptually different from "converting the habitability of Earth into dollar bills" or "converting the health problems of human beings into a maximal amount of dollar bills", which is precisely what many corporations _actually do_? The "corporation as AI" metaphor isn't about some abstract future possibility, it's an explanatory mechanism for…

Yeah, I agree. My original post was bringing up what I thought was an interesting comparison between hypothetical software AI and the corporations-as-AI metaphor presented in the talk. Corporations _are_ acting as misaligned optimizers. Solving that problem is hugely important. However, the "AI" comparison breaks down somewhat when you start thinking about how we might actually fix the problem. With corporations, we…

I don't know that we won't have tools, albeit different ones, to regulate a bad AI. AI need more than just intelligence and agency. They also need effective ways to interact and affect their environment. That boundary is where we are likely to develop tools to limit and regulate them.

If they are truly general AI then it's likely that their reaction to that limitation and regulation will be not dissimilar to a person's but I see no reason to assume that limiting them will be impossible.

Re: Dude, You Broke the Future (2017)

#48

Post by dodobirdlord »

ALittleLight wrote: post #19

I found a couple errors early on that dissuaded me from continuing to read. One was the idea that the author could rule out the singularity because he wasn't aware of progress toward self-motivated AI that would be something like our own intelligence. This seems like a limited view to me because we wouldn't need self-motivation at all in AI to hit the singularity. Humans can supply the self motivation. Suppose I'm us…

> It seems like the author kind of misses the point of the paperclip thought experiment, which is, broadly, that an AI's interests might not be aligned with our own and that misalignment may cause serious problems.

Perhaps you should have finished reading. What you suggest the author missed is essentially the core thesis of the piece.

Re: Dude, You Broke the Future (2017)

#49

Post by worldsayshi »

ALittleLight wrote: post #19

I found a couple errors early on that dissuaded me from continuing to read. One was the idea that the author could rule out the singularity because he wasn't aware of progress toward self-motivated AI that would be something like our own intelligence. This seems like a limited view to me because we wouldn't need self-motivation at all in AI to hit the singularity. Humans can supply the self motivation. Suppose I'm us…

Another point: While not all corporations are profit maximizers in a paperclip-maximizer obsessive way - those corporations that grow largest and consume others tends to be those that obsess most about profit and growth simply because those are the ones that tend to become big.

Re: Dude, You Broke the Future (2017)

#50

Post by Stierlitz »

ALittleLight wrote: post #19

I found a couple errors early on that dissuaded me from continuing to read. One was the idea that the author could rule out the singularity because he wasn't aware of progress toward self-motivated AI that would be something like our own intelligence. This seems like a limited view to me because we wouldn't need self-motivation at all in AI to hit the singularity. Humans can supply the self motivation. Suppose I'm us…

@ALittleLight: “.. One was the idea that the author could rule out the singularity ..”

Your consciousness uploaded and then downloaded into a synthetic human like body isn't ever going to be you. As the man said “transhumanism is a warmed-over Christian heresy”.

Post reply on HN