Live data from Hacker News

Should There Be Limits on Persuasive Technologies?

schneier.com

71–80 of 241 posts

Re: Should There Be Limits on Persuasive Technologies?

#71
post #48

I wrote this for a related article yesterday: --- If we know that humans have all sorts of cognitive biases, how come it's ok to use that fact while at the same time we insist there's some kind of free market? Say you discover that putting good-looking women next to cars causes the sale of cars to increase. Why does nobody question whether it is legitimate to do so? It's as if there's a line between actively lying ("…

That point is sort-of pointless. It would be fair to boil a lot of the argument there down to "People don't act randomly. They have reasons for what they are doing. I think their reasons are bad". I can't argue with that, but the alternatives are worse. If you centralise power, sooner or later the advertising exec gets control of the powerful body, and now you can't choose to resist even if you can see that what is h…

You could deploy the same argument in favor of allowing any kind of fraudulent product or service; the full libertarian "caveat emptor" approach. However, not only is this unpopular, prohibiting fraud doesn't in and of itself result in fraudsters taking over the market.

The boundary between overly enthusiastic promises and actual fraud is one that's in different places for different jurisdictions and is constantly at the forefront of litigation as people invent new fraud schemes.

> "People don't act randomly. They have reasons for what they are doing. I think their reasons are bad".

Sometimes the reasons are bad in very objective ways.

A goes down to the market to buy a kilogram of apples. A vendor B advertises 1kg apples for a pound. He weighs them out on his scales. A hands over his pound. When he gets home he finds he has 800g of apples. Was A's reason for purchasing the apples from B good or bad?

(laws against short measure have been a thing since at least Roman times; I believe they also had a few product quality laws, although the canonical example there is always German beer law from 1516)

Re: Should There Be Limits on Persuasive Technologies?

#72
post #9

Earlier quoted context omitted.

My point is that I think this kind of thinking is disempowering and counterproductive. By framing technology companies as all powerful, even if that's possibly inaccurate and overblown, tech supremacy becomes inevitable in the eyes of people who see it as lacking any alternatives. ("If they can predict me this well, they must know better, right?") I think this is why Silicon Valley loves to invite their critics who a…

Why can't both be true though For example, regression models aren't exactly ai magic, but unchecked, they compute discriminatory insurance premiums in a way we find morally unacceptable. The naked emperor still needs keeping in check.

(addendum: actually, I can't help but feel that although it's been around since before I was born, I fully understand how it works and use it on a regular basis, regression still is ai magic ;-) )

Re: Should There Be Limits on Persuasive Technologies?

#73
I said it before I will say it again:

The new thing, and the key point here is scale.

You can discuss persuasion or cognitive bias away until everyone gets an aneurysm, sure. Some example: You can't not communicate. So even not persuading is persuading. What if the presidential candidate would just not give a scheduled speech? How do you communicate information objectively? Casual language creates bias, so does scientific language, simple language, passive language, active language. "Cognitive Bias" might as well be called "Cognition", since it is just how the brain works. You have to think "tree" immediately when you see one, even before you validated that all leaves are real and that the whole thing is not a projection on an transparent screen. Otherwise you can't function.

But: Big tech throws us in a situation where a small group of people influences our perception on a massive scale. Facebook changes a sentence on their homepage and a billion people read it. Youtube raises some parameter (yeah I know that's not how AI works) by 0.01 and the political opinion about the Grenfell tower disaster changes ever so slightly - for 30 million people. Google's filter has a tiny hole and some troll broadcasts wrong medical information about gout to 200k people.

Every time one of these things happen, the world shakes. Dozens die or survive. Demonstrations form and elections swing. Opportunities are wasted and ideas surface.

I am not arrogant to actually propose an easy solution to this, and I don't think there is one. Just be aware that "I can always go and stab someone" is not a good argument when you are discussing a fully automated drone swarm with kill authority.

Re: Should There Be Limits on Persuasive Technologies?

#74

I said it before I will say it again: The new thing, and the key point here is scale . You can discuss persuasion or cognitive bias away until everyone gets an aneurysm, sure. Some example: You can't not communicate. So even not persuading is persuading. What if the presidential candidate would just not give a scheduled speech? How do you communicate information objectively? Casual language creates bias, so does scie…

To your point about scale, once a tweak is made and X or Y "news story" is let loose in the wild, it gets amplified. The scale of the impact isn't linear, it's something more exponential.

Re: Should There Be Limits on Persuasive Technologies?

#75

I wrote this for a related article yesterday: --- If we know that humans have all sorts of cognitive biases, how come it's ok to use that fact while at the same time we insist there's some kind of free market? Say you discover that putting good-looking women next to cars causes the sale of cars to increase. Why does nobody question whether it is legitimate to do so? It's as if there's a line between actively lying ("…

It's an interesting discussion.

Of course I don't think blandifying ads is the way to go. Making ads attractive is a cultural phenomenon.

An implicit suggestion of fame, class, social status is fine. It's part of the game. It's an implicit suggestion in the end.

But everything has a limit. What might be acceptable in an adult ad might not be so acceptable for a teenager or a kids ad.

More worrying is propaganda, which is misleading/false in non-obvious ways and not explicitly an ad.

Re: Should There Be Limits on Persuasive Technologies?

#76

I said it before I will say it again: The new thing, and the key point here is scale . You can discuss persuasion or cognitive bias away until everyone gets an aneurysm, sure. Some example: You can't not communicate. So even not persuading is persuading. What if the presidential candidate would just not give a scheduled speech? How do you communicate information objectively? Casual language creates bias, so does scie…

Isn’t this a bit like mode collapse in GANs?

Re: Should There Be Limits on Persuasive Technologies?

#77
post #52

I wrote this for a related article yesterday: --- If we know that humans have all sorts of cognitive biases, how come it's ok to use that fact while at the same time we insist there's some kind of free market? Say you discover that putting good-looking women next to cars causes the sale of cars to increase. Why does nobody question whether it is legitimate to do so? It's as if there's a line between actively lying ("…

As an absurd example, if someone were to invent a ray-gun that reprograms people's brains with arbitrary beliefs it would be 100% unethical and illegal to use. However, ads or fake news article that takes many exposures to reprogram you (by using the cognitive bias back channels as you mention) are perfectly fine. ¯\_(ツ)_/¯

With the raygun there's a clear cause and effect. You can fairly easily prove that what changes beliefs is the raygun. You can't do that with ads and fake news articles.

The other point is that with ads and articles you have to engage with them yourself. Something can grab your attention, but you can decide to ignore it anyway.

Re: Should There Be Limits on Persuasive Technologies?

#78
post #42

Earlier quoted context omitted.

Some economists have tried arguing that advertising adds value to products, but I think it only hinders people from making choices based on the product's merit. It's a distortion of the market. We wouldn't lose much if advertising would just be forbidden. A traditional definition of it would probably be enough to get rid of most of it. The main problem is that huge sums of money are involved, and almost all media pro…

It would seem that review sites would be a massive beneficiary of a ban on advertising; whether or not they review in a Consumer Reviews (scientifically backed, generally) or Bob’s Internet Affiliate Shills or Celebrity Chris’ Favorites manner is left as an exercise to the reader.

At least review sites would be something that a user would have to seek out, rather than having the information blasted into their field of attention at times the advertisers found strategically optimal.

Also, if people found themselves regretting purchases of products and services recommended to them by a particular review site, they would be free to use a different review site in future.

Hopefully these sites wouldn't operate on a non-refundable yearly-subscription model, making people reluctant to "waste" the money they had spent. The business model for such sites might be a little tricky, since they shouldn't really be making money from affiliate fees, and web advertising wouldn't exist in this world either.

Re: Should There Be Limits on Persuasive Technologies?

#79

I wrote this for a related article yesterday: --- If we know that humans have all sorts of cognitive biases, how come it's ok to use that fact while at the same time we insist there's some kind of free market? Say you discover that putting good-looking women next to cars causes the sale of cars to increase. Why does nobody question whether it is legitimate to do so? It's as if there's a line between actively lying ("…

The wider culture is supposed to counterbalance for this.

Taking things to the extreme: maybe people are by default violent and will kill a few people in their lifetime to get their way. But the culture counterbalances this.

Likewise everyone old enough to have a sex drive has seen enough publicity with barely-clad attractive women to know the ruse. Of course, people are still vulnerable to these things (cf. onlyfans, etc.) but at some point you have to establish that the rules are clear and if people still want to indulge the fantasy of a beautiful woman by buying a car, well, whatever.

Sure, we still put backstops to this with drugs, gambling. But the use of sex as a tool for attention grabbing is way too wide a net to cast.

Re: Should There Be Limits on Persuasive Technologies?

#80
post #42

I wrote this for a related article yesterday: --- If we know that humans have all sorts of cognitive biases, how come it's ok to use that fact while at the same time we insist there's some kind of free market? Say you discover that putting good-looking women next to cars causes the sale of cars to increase. Why does nobody question whether it is legitimate to do so? It's as if there's a line between actively lying ("…

Some economists have tried arguing that advertising adds value to products, but I think it only hinders people from making choices based on the product's merit. It's a distortion of the market. We wouldn't lose much if advertising would just be forbidden. A traditional definition of it would probably be enough to get rid of most of it. The main problem is that huge sums of money are involved, and almost all media pro…

>We wouldn't lose much if advertising would just be forbidden.

We would lose the current ad industry and just replace it with some other form of it.

You can't get rid odd advertising, because the existence of the product itself is already advertising. Let's imagine that advertising were banned. How could a car manufacturer still advertise their cars? Put bigger logos on them and sell lots of cheaper models for a while. That way the city will be full of your vehicles and any time someone sees a car they'll think of your cars. Same goes for Coca-Cola and other brands. In that case they just need to take up more shelf-space.

Post reply on HN