Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are you so sure I'll lose?

twitter.com

Post reply on HN
Print view View original

61–70 of 90 posts

Re: Why are you so sure I'll lose?

#61

Post by eigenket »

pointlessone wrote: post #57
eigenket wrote: post #13

Earlier quoted context omitted.

In addition to a hypothetical AI, it sounds like you are describing real companies here.

You're not wrong but also real companies consist of people. Individually they share many of their thinking patterns and moral views with people outside of those companies. So even though collective nature of companies diffuse responsibility and somewhat erode moral considerations of collective action it's relatively hard to come up with Paperclip Maximizer LLC. Eventually individual morality will kick in and there wi…

>Eventually individual morality will kick in and there will be a revolt against dismantling of the Earth.

This does not seem to be a given. Indeed one can look at the fossil fuel industry for what looks like counterexamples.

Re: Why are you so sure I'll lose?

#62

Post by lucubratory »

xcode42 wrote: post #54

While I find this story a fun and intelligent allegory for the issue of ASIs and a fun thought experiment, I find all the doomsaying around ai safety research kind of tiring. AI safety research is an important and worthwhile field without us needing to constantly be worrying about a far off doomsday that may never come. Yes ASIs are an existential threat and yes once they are out, they are out, and there's no stoppin…

What specific demonstrations of capability by AI would convince you that your proposed timeline for reaching AGI isn't correct, and that it will happen significantly sooner? If it helps to think about it, there's a good DeepMind paper on a relatively concrete scaling system on the path to AGI: https://arxiv.org/abs/2311.02462

Re: Why are you so sure I'll lose?

#63

Post by jacquesm »

aerojoe23 wrote: post #58
jacquesm wrote: post #47

Earlier quoted context omitted.

Rock climbing = personal entertainment. That's why rock climbers don't need to worry about Stockfish. But competitive chess players do have to worry about Stockfish.

There is competitive rock climbing. Rock climbing isn't all about physical strength, you have to pick your route. Eventually some military robot company will want to prove that its combat bot can out perform humans on any terrain. And we'll see best humans vs matches that can match and a few years later beat their performance. Of course the professional competitors don't have to worry about competing against a human…

I like to keep my science fiction and real world concerns firewalled off from each other. I sleep bad enough as it is.

Re: Why are you so sure I'll lose?

#64

Post by lucubratory »

ZeroGravitas wrote: post #53

Does skipping the 80 years of computer chess development leading up to stockfish make this a worse or better analogy? I don't quite understand it but the group of people saying "AI is an evil god that will destroy us all" and the group of people saying "I am building an even stronger and more powerful AI with an explicit goal of it becoming godlike" seem to overlap more than you'd think.

You're talking about the e/acc people like Beff Jezos who want humanity to be eradicated by the AI god they want to summon? It's only a contradiction if you assume they want themselves and humanity in general to stay alive or have any significant decision making power going forward. That assumption is not correct, they want the AI to have power and are either indifferent or actively hostile to humans surviving the experience.

Re: Why are you so sure I'll lose?

#65

Post by pointlessone »

eigenket wrote: post #61
pointlessone wrote: post #57

Earlier quoted context omitted.

You're not wrong but also real companies consist of people. Individually they share many of their thinking patterns and moral views with people outside of those companies. So even though collective nature of companies diffuse responsibility and somewhat erode moral considerations of collective action it's relatively hard to come up with Paperclip Maximizer LLC. Eventually individual morality will kick in and there wi…

>Eventually individual morality will kick in and there will be a revolt against dismantling of the Earth. This does not seem to be a given. Indeed one can look at the fossil fuel industry for what looks like counterexamples.

It is not a counterexample. Most individuals are having hard time taking into account consequences that are decades removed and either vaguely defined (e.g. 3℃ global temperature raise is a catastrophe) or too great in scope and scale (e.g. literally global societal collapse). But even so externalities are catching up. British Petroleum is pivoting into solar, for instance even if for PR reasons. As a result of external pressure (public opinion and legislation/regulation) fossil fuel industry is in decline. Most major companies are trying to diversify and do something else instead. Companies that comprise fossil fuel industry will probably remain but they'll do something else.

You can not be sure you can shame or legislate paperclip maximizer, though. An average paperclip maximizer has no shame and doesn't care for any law.

Re: Why are you so sure I'll lose?

#66

Post by eigenket »

pointlessone wrote: post #65
eigenket wrote: post #61

Earlier quoted context omitted.

>Eventually individual morality will kick in and there will be a revolt against dismantling of the Earth. This does not seem to be a given. Indeed one can look at the fossil fuel industry for what looks like counterexamples.

It is not a counterexample. Most individuals are having hard time taking into account consequences that are decades removed and either vaguely defined (e.g. 3℃ global temperature raise is a catastrophe) or too great in scope and scale (e.g. literally global societal collapse). But even so externalities are catching up. British Petroleum is pivoting into solar, for instance even if for PR reasons. As a result of exter…

Whether or not BP works as a counter example has no bearing on other examples. Aramco and ExxonMobil certainly look like a counterexamples to me.

Re: Why are you so sure I'll lose?

#67

Post by pointlessone »

chmod775 wrote: post #29
jacquesm wrote: post #7

So, the solution is not to play on a battlefield where the AI has an advantage. Instead play with humans and have fun. Meanwhile all battlefields will one-by-one become battlefields where the AI has the advantage. Because in the end almost every contest between humans is either one based on strength or on speed and the AI will always be able to match us for speed. And strength has long been the domain of machines, co…

> And strength has long been the domain of machines, couple machines with AI and you lose all of these as well and you lose them faster. I don't think rock climbers need to worry about Stockfish. We're going to lose to specialized machines every time while putting to shame any general purpose machines of our own creation. At least for a while yet no machine is going to match the breadth of our capabilities and our se…

Say, you get a super-human chess AI, and a super-human accountant AI, and a super-human medical doctor AI, and a super-human material design AI, and a super-human lawyer AI, and… How many of these do you need in a bunch when it's going to be better than, most people at most things? You say that people are great generalists but what if there going to be a specialised "conductor" AI that is super-human at establishing communication between specialised AIs and dividing tasks super-humanly efficiently between them?

So when to start worrying?

Re: Why are you so sure I'll lose?

#68

Post by 303uru »

endtime wrote: post #32

We need a catalog of simple refutations of bad arguments and common misconceptions about AGI risk, with a short domain like agi.lol, so we can respond to comments like "I bet the guy who has spent 20 years worrying about AGI didn't think of this" with a simple link like agi.lol/107 or agi.lol/the-human-can-have-an-ai-too.

Really channeling that inner kid.

Re: Why are you so sure I'll lose?

#69

Post by mitthrowaway2 »

jacquesm wrote: post #49
dragonwriter wrote: post #48

Earlier quoted context omitted.

> AI doesn't act by itself It can be made to quite easily. > and worst case you could unplug the bloody thing Not necessarily. > But the guys with the guns are the same guys with the guns Well, unless they are autonomous AI drones with guns (or missiles), in which case they are different “guys” with guns.

Sure, I've seen those movies. But today it's the guys with the guns/drones/grenades/missiles. I'm worried about stable dictatorships far more than I am about AGI taking over the world and offing humanity because stable dictatorships are already using every weapon they can lay their hands on to control the population and AGI might just tip the scale to the point where they become eternal. And regular AI (the kind that…

> But today

There's no question that this is a tomorrow-focused concern, just like climate change or Kessler syndrome, et cetera.

And today's concerns are important too. Nobody is dismissing them. But tomorrow comes quite quickly these days.

You're allowed to be concerned about both.

Re: Why are you so sure I'll lose?

#70

Post by mitthrowaway2 »

jacquesm wrote: post #46
mitthrowaway2 wrote: post #16

Earlier quoted context omitted.

The above sentiment is posted many times on every such thread, it would be greatly enriched with am explanation of why you're not worried about the AI. Because worrying about B is not a rationale for not worrying about A.

AI doesn't act by itself and worst case you could unplug the bloody thing (SF notwithstanding). But the guys with the guns are the same guys with the guns that have been in charge for as long as there are guns (with some rare exceptions) and I really don't think the existence of AI will change human nature. So AI will be used as a weapon long before it will be a standalone threat. And I like to focus on the things th…

> worst case you could unplug the bloody thing

Okay, now you're failing at putting yourself in the AI's shoes and asking how to win from its position, even using your own intelligence.

Have you even considered how you might win as an AI, knowing that you can be unplugged? It's definitely doable.

Post reply on HN