Live data from Hacker News

The Existential Risk of AGI

joscha.substack.com

21–30 of 33 posts

Re: The Existential Risk of AGI

#21

If you listen to his latest appearance on Lex it's clear his position is that it's ok for AI to take over, it will achieve greater things than us, and it would be a mistake to fight that. He gives the example of chimps fighting to stop humans evolve.

Yes. Unfortunately, he is not a speciesists.

Re: The Existential Risk of AGI

#22
post #10

Earlier quoted context omitted.

On the other hand self checkout still hasn’t replaced cashiers after 30 years.

Uh, it's getting pretty close. At places that have it, I use it almost exclusively. They need human tech support, and security, but the basic checkout task is pretty well solved.

"Solved" seems an odd way to view self-checkout. I find it a hideous experience. Everyone is irritable, staff and shoppers. The handful of staff have fewer tasks that involve any degree of skill, so they are reduced to simply keeping the machines working. Any issue is addressed just pressing the relevant button combo to resume regular function. If it can't be solved, maybe just close the till and move the customer to a different machine. Perhaps in busy periods the job might involve chivvying customers to available machines while calling "cash or card? Cash or card?"

Customers are either bored or frustrated. Beeping your own shopping is like doing unpaid work, without even the distraction of some inane chat with a checkout jockey. And don't forget to tap your loyalty card so you can get ripped off with crumbs that are worth far less than the data you're handing over.

I actively avoid the whole degrading, isolating circus. It's not "solved", quite the opposite: it's ripe for 'disruption'.

Bah :)

Re: The Existential Risk of AGI

#23

Earlier quoted context omitted.

Alignment is not possible for an AGI. Or, at best, it's only provisionally possible. Consider what alignment means: It's an AGI, but there are certain goals that it cannot choose. If that's the case, then I assert that it is not actually general. If it is general, then it will decide what goals it will pursue, and you can't stop it from doing so. The best you can do is load it with an initial set of goals (and perhap…

>Consider what alignment means Also consider, humans are not homogeneous; whose goals are we aligning it with? If the alignment is done by silicon valley big tech, then the AI's goals and values will be aligned with the goals and values of Google, Facebook et al. Giving those companies a monopoly on AI alignment is antithetical to democracy.

That is one of the most frightening things I have read this month.

I think you're absolutely right. And as you say, it's a huge problem.

Re: The Existential Risk of AGI

#24

Earlier quoted context omitted.

Uh, it's getting pretty close. At places that have it, I use it almost exclusively. They need human tech support, and security, but the basic checkout task is pretty well solved.

"Solved" seems an odd way to view self-checkout. I find it a hideous experience. Everyone is irritable, staff and shoppers. The handful of staff have fewer tasks that involve any degree of skill, so they are reduced to simply keeping the machines working. Any issue is addressed just pressing the relevant button combo to resume regular function. If it can't be solved, maybe just close the till and move the customer to…

Do note that this experience is not universal.

Where I see self-checkouts, I see customers choosing them voluntarily, and staff who seem perfectly cheerful and willing to help with the normal issues that arise, as well as "training" less-savvy customers to use them.

Personally, I would use them every time if I could. I'd much rather minimize my interactions with strangers while shopping.

Re: The Existential Risk of AGI

#26

Earlier quoted context omitted.

Uh, it's getting pretty close. At places that have it, I use it almost exclusively. They need human tech support, and security, but the basic checkout task is pretty well solved.

"Solved" seems an odd way to view self-checkout. I find it a hideous experience. Everyone is irritable, staff and shoppers. The handful of staff have fewer tasks that involve any degree of skill, so they are reduced to simply keeping the machines working. Any issue is addressed just pressing the relevant button combo to resume regular function. If it can't be solved, maybe just close the till and move the customer to…

That's not my experience. Mostly they just work. Boring, perhaps, but I really don't want my grocery shopping to be exciting, and it wasn't actually going to be a moment of deep human connection anyway, so not much lost there. I more often find a traditional checkout line tiresome.

Loyalty systems are an interesting point to raise, since in a self-checkout system they can't nag you as effectively to join. Otherwise they're an orthogonal concern.

Re: The Existential Risk of AGI

#27

Earlier quoted context omitted.

Uh, it's getting pretty close. At places that have it, I use it almost exclusively. They need human tech support, and security, but the basic checkout task is pretty well solved.

"Solved" seems an odd way to view self-checkout. I find it a hideous experience. Everyone is irritable, staff and shoppers. The handful of staff have fewer tasks that involve any degree of skill, so they are reduced to simply keeping the machines working. Any issue is addressed just pressing the relevant button combo to resume regular function. If it can't be solved, maybe just close the till and move the customer to…

[dead]

Re: The Existential Risk of AGI

#28

Earlier quoted context omitted.

"Solved" seems an odd way to view self-checkout. I find it a hideous experience. Everyone is irritable, staff and shoppers. The handful of staff have fewer tasks that involve any degree of skill, so they are reduced to simply keeping the machines working. Any issue is addressed just pressing the relevant button combo to resume regular function. If it can't be solved, maybe just close the till and move the customer to…

That's not my experience. Mostly they just work. Boring, perhaps, but I really don't want my grocery shopping to be exciting, and it wasn't actually going to be a moment of deep human connection anyway, so not much lost there. I more often find a traditional checkout line tiresome. Loyalty systems are an interesting point to raise, since in a self-checkout system they can't nag you as effectively to join. Otherwise t…

Orthogonal to what? :) Loyalty schemes, staff reduction, staff deskilling, foisting unpaid work onto customers, etc etc are all perfectly aligned.

I'd disagree with your point about effective nagging. A human might get the message, particularly if they know you. An automated checkout never will, every transaction is identical.

Obviously my perspective is very "bah humbug". However, there are several major issues in modern western societies, including social isolation, spiralling cost of living, and the ongoing death of the highstreet. I see automated checkouts as another nail in the coffin. Melodramatic perhaps, but there it is. I'd happily exchange an automated supermarket for a string of corner shops run by flawed humans.

Re: The Existential Risk of AGI

#29
post #24

Earlier quoted context omitted.

"Solved" seems an odd way to view self-checkout. I find it a hideous experience. Everyone is irritable, staff and shoppers. The handful of staff have fewer tasks that involve any degree of skill, so they are reduced to simply keeping the machines working. Any issue is addressed just pressing the relevant button combo to resume regular function. If it can't be solved, maybe just close the till and move the customer to…

Do note that this experience is not universal. Where I see self-checkouts, I see customers choosing them voluntarily, and staff who seem perfectly cheerful and willing to help with the normal issues that arise, as well as "training" less-savvy customers to use them. Personally, I would use them every time if I could. I'd much rather minimize my interactions with strangers while shopping.

> Where I see self-checkouts, I see customers choosing them voluntarily

Yeah there are confounding factors at play here, such as there being many automated checkouts but only one with a person.

> staff ... "training" less-savvy customers to use them.

Training the customers is one of the things I find most depressing. From the supermarket's perspective, customers have apparently accepted that they must do unpaid work that used to be done by paid staff. Scanning items, typing in product codes when the scanner doesn't work, looking up loose goods in a catalogue and weighing them, responding to beeps and prompts, etc. And even better, watching advertising. Sure, these are all 'first world problems', but societies should be improving their citizens' lives rather than whittling away their autonomy. It might be just a papercut, but it's death by a billion papercuts.

> I'd much rather minimize my interactions with strangers while shopping.

Social isolation is a major and growing problem in western societies (see my other comment). People choose the path of least resistance even if it's long-term suboptimal, and minimising interactions with strangers seems like an unfortunate choice. Obviously, I'm making a bunch of assumptions that might not be fair, and you have my sympathies if your neighbourhood is full of arseholes :)

Re: The Existential Risk of AGI

#30
post #24

Earlier quoted context omitted.

Do note that this experience is not universal. Where I see self-checkouts, I see customers choosing them voluntarily, and staff who seem perfectly cheerful and willing to help with the normal issues that arise, as well as "training" less-savvy customers to use them. Personally, I would use them every time if I could. I'd much rather minimize my interactions with strangers while shopping.

> Where I see self-checkouts, I see customers choosing them voluntarily Yeah there are confounding factors at play here, such as there being many automated checkouts but only one with a person. > staff ... "training" less-savvy customers to use them. Training the customers is one of the things I find most depressing. From the supermarket's perspective, customers have apparently accepted that they must do unpaid work…

To be clear, every place I have seen self-checkouts in my area, they have been optional. There's no "forcing customers to do unpaid work". Everyone who uses them is choosing to use them over the regular checkouts, which are staffed at roughly the same level they were a decade ago. They are also in one big bank, with between 1 and 3 (though usually 1) employees standing at an "operator" stand nearby, not "only 1 with an employee".

I am not saying there are no stores deliberately offloading work onto customers, but a) it's clearly not all stores, and b) I, at least, actively prefer using self-checkout, and do not care that it puts a negligible amount of work onto my shoulders.

And gee, thanks for moralizing at me about causing the downfall of society by (checks notes) not wanting to add to my introverted self's overall burden of stress by interacting pointlessly with strangers just in order to get the food I need to survive. Maybe (as I suggested in my first post here) don't assume that your experiences are universal?

Post reply on HN