Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Europe seeks to limit use of AI in society

bbc.com

Post reply on HN
Print view View original

111–120 of 154 posts

Re: Europe seeks to limit use of AI in society

#111

Post by someguy321 »

tomp wrote: post #103
joe_the_user wrote: post #94

The listed examples seem reasonably good. * systems which establish priority in the dispatching of emergency services * systems determining access to or assigning people to educational institutes * recruitment algorithms * those that evaluate credit worthiness * those for making individual risk assessments * crime-predicting algorithms While I'd also like to see autonomous military devices banned, banned AI that make…

> we've seen ways AI can allow discrimination through the back door. Do you have any concrete examples of these, in particular where the use of statistics or AI enables more discrimination than using human decisions?

One I can think of off the top of my head (statistics, not AI, although AI would also allow it) is that the actuarial calculations for home/car insurance quotes rely on risk data by zip code, education level, income, and any and all other socioeconomic variables not including protected class, but which often correlate/group by protected class, and which are also reliable indicators of risk.

Depending on who you talk to these algorithms either are or are not discriminating against protected classes "through the back door".

Re: Europe seeks to limit use of AI in society

#112

Post by joe_the_user »

trutannus wrote: post #62
geraneum wrote: post #56

Earlier quoted context omitted.

Government (ab)use of AI is is of course a serious threat. But I’d say big corporations abuse of AI is even worse. Assuming we are talking about a democratic state, at least there are some checks and balances on governments whereas people cannot elect a FAANG CEO or go to a ‘.gov’ website to read a transcript of board meetings. Edit: I am by no means advocating for government’s use of AI in any form.

A business can't lock you in a prison cell. I'd argue that the checks and balances at this point are little more than a mirage. Ruling by fiat is becoming more common, accountability less so. Government use of AI is far more menacing to me than a business using it.

Ruling by fiat is becoming more common, accountability less so.

Well, PG&E cut down a whole bunch of trees in my town just recently, against the objections of the locals and the city council. And Judge Alsop, who supervises their bankruptcy chided them for this slap-dash, crude effort to show they were doing thing (didn't stop them, darn it).

So you see a bunch of actions that look like the state or industry acting by fiat. But it only looks that way. The many institutions of this society are at loggerheads with each other, the parties are in gridlock, etc. The main thing is they've shut out the average person from their debates - which is a bit different.

Re: Europe seeks to limit use of AI in society

#113

Post by tomp »

YeGoblynQueenne wrote: post #110
tomp wrote: post #103

Earlier quoted context omitted.

> we've seen ways AI can allow discrimination through the back door. Do you have any concrete examples of these, in particular where the use of statistics or AI enables more discrimination than using human decisions?

Why does it have to be "more"?

Parent mentioned that AI is used to sneak in discrimination through the back door, implying that discrimination wouldn’t be there (or there would be less) without AI.

Re: Europe seeks to limit use of AI in society

#114

Post by Aerroon »

daenney wrote: post #3
keiferski wrote: post #2

Bonus points if they name the overseeing organization the Turing Police. And if Switzerland goes its own way and allows citizenship for AIs. In all seriousness, I’m not sure if these legal restrictions will actually be effective. They are too broad, vague, and will likely just result in technological stagnation.

Laws aren’t immutable and right now this is just a proposal. We’ll have to see what its final form ends up being. > will likely just result in technological stagnation. Nothing in this stops any kind of research nor does it ban its use. It just limits how effortlessly you can invade people’s privacy and discriminate against them. It may very well help research in underinvested areas of AI, and it’s ethical consequenc…

>Laws aren’t immutable

You could've fooled me. About the only thing that seems to change laws in places like the EU is when the courts decide to strike something down. Everything else seems to just go ahead the way the EU politicians envisioned. Consequences be damned.

Re: Europe seeks to limit use of AI in society

#115

Post by tomp »

someguy321 wrote: post #111
tomp wrote: post #103

Earlier quoted context omitted.

> we've seen ways AI can allow discrimination through the back door. Do you have any concrete examples of these, in particular where the use of statistics or AI enables more discrimination than using human decisions?

One I can think of off the top of my head (statistics, not AI, although AI would also allow it) is that the actuarial calculations for home/car insurance quotes rely on risk data by zip code, education level, income, and any and all other socioeconomic variables not including protected class, but which often correlate/group by protected class, and which are also reliable indicators of risk. Depending on who you talk…

Sure but my point is that, while you could argue that decisions about some topics could be discriminatory by definition, that has nothing to do with AI (and saying that AI is at fault is pure anti-AI FUD).

Re: Europe seeks to limit use of AI in society

#116

Post by tablespoon »

tomp wrote: post #103
joe_the_user wrote: post #94

The listed examples seem reasonably good. * systems which establish priority in the dispatching of emergency services * systems determining access to or assigning people to educational institutes * recruitment algorithms * those that evaluate credit worthiness * those for making individual risk assessments * crime-predicting algorithms While I'd also like to see autonomous military devices banned, banned AI that make…

> we've seen ways AI can allow discrimination through the back door. Do you have any concrete examples of these, in particular where the use of statistics or AI enables more discrimination than using human decisions?

>>> we've seen ways AI can allow discrimination through the back door.

>> we've seen ways AI can allow discrimination through the back door.

> Do you have any concrete examples of these, in particular where the use of statistics or AI enables more discrimination than using human decisions?

Here's a concrete example:

https://towardsdatascience.com/racist-data-human-bias-is-inf...

> An AI program called COMPAS has been used by a Wisconsin court to predict the likelihood that convicts will reoffend. An investigative piece by ProPublica last year found that this risk assessment system was biased against black prisoners, incorrectly flagging them as being more likely to reoffend than white prisoners (45% to 24% respectively). These predictions have led to defendants being handed longer sentences, as in the case of Wisconsin v. Loomis.

Also, I'd dispute your injection of the "more racist than humans" framing (which is also moving the goalposts a bit). The problem with racist algorithms isn't necessarily their "degree of racism" but the fact that they mask very real racism behind a veneer of false computerized "objectivity."

Re: Europe seeks to limit use of AI in society

#117

Post by YeGoblynQueenne »

avz wrote: post #63
8fGTBjZxBcHq wrote: post #7

Earlier quoted context omitted.

My perspective is that our technological advancement has well outpaced our ability to adapt to the changes or bring our legal and social tools effectively to bear on them. A decade or two of stagnation would be frustrating for those in the field but probably overall a good thing. Plus I don't think this would affect research at all so not even.

> A decade or two of stagnation would be frustrating for those in the field but probably overall a good thing. Is a decade or two of a head start given to high-tech totalitarian regimes like China overall a good thing? > Plus I don't think this would affect research at all so not even. Limiting use of AI reduces interest of the public and young researchers and engineers, contributes to brain drain and limits availabi…

>> Limiting use of AI reduces interest of the public and young researchers and engineers, contributes to brain drain and limits availability of large datasets that are an important asset for AI development.

I disagree. As a senior-year PhD student, I am relieved that the EU is taking a stance on this matter and hope that others in the West will follow suit (it's probably too late for China). I am relieved because I personally have grave concerns about the uses of AI in society and have thought for some time that some kind of formal and official framework is needed. AI researchers haven't yet managed to establish such a framework, so legislators have stepped in. The framework still seems pretty "green" and like it will take a lot of development and improvement, but that a first step was made is important.

So in fact you might say that having a legal framework in place makes AI research more attractive, because the student is not left to wonder about the ethics of her research on her own.

As to the availability of large datasets- how do you see that this would be affected by the legislation being considered?

I should also point out that the reliance on large datasets is a bug, not a feature, of the currently dominant AI techniques and that an alternative is sorely needed. If large datasets became less easily available, that would give a good incentive to researchers to go do something new, rather than throw a ton of data to an old benchmark to improve it by 0.2%.

Re: Europe seeks to limit use of AI in society

#118

Post by neilparikh »

joe_the_user wrote: post #94

The listed examples seem reasonably good. * systems which establish priority in the dispatching of emergency services * systems determining access to or assigning people to educational institutes * recruitment algorithms * those that evaluate credit worthiness * those for making individual risk assessments * crime-predicting algorithms While I'd also like to see autonomous military devices banned, banned AI that make…

I think the tradeoff is that at least the AI discrimination is systemized, and there's one place you can manipulate to reduce that discrimination, while with pre-AI human discrimination, it's not at one place, so it's harder to eliminate.

As an example, it's the difference between being rejected by a central agent for a loan, versus going to your local branch, and being rejected by a random employee at the local branch. It's obviously much easier to change the central agent than it is to change every distributed employee.

Now, whether this is actually the case in practice, and whether this is a good or bad thing is open to interpretation.

Re: Europe seeks to limit use of AI in society

#119

Post by roomey »

It's been in Irish law for some time (which makes it applicable to the majority of US tech giants). Unfortunately Ireland isn't hq for a lot of the big finance and insurance firms which would probably be more useful in this situation. Anyway, here is the latest version of the law: TLDR; if it impacts you it should require a human decision and be appeal-able. Also there is a right to review code but I'm not sure how that works:

the right of a data subject not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her shall, in addition to the grounds identified in Article 22(2)(a) and (c), not apply where—

(a) the decision is authorised or required by or under an enactment, and

(b) either—

(i) the effect of that decision is to grant a request of the data subject, or

(ii) in all other cases (where subparagraph (i) is not applicable), adequate steps have been taken by the controller to safeguard the legitimate interests of the data subject which steps shall include the making of arrangements to enable him or her to—

(I) make representations to the controller in relation to the decision,

(II) request human intervention in the decision-making process,

(III) request to appeal the decision.

Re: Europe seeks to limit use of AI in society

#120

Post by avz »

avz wrote: post #100
imapeopleperson wrote: post #52

No one actually wants a society being ruled by computers, except maybe the people running those computers. This is at least a step in the right direction.

> No one actually wants a society being ruled by computers, except maybe the people running those computers. Humans in charge are known for injustice, kickbacks, favor trading, selective enforcement and other forms of corruption and abuse. Properly engineered and regularly reviewed open source systems with balance checks might just get us closer to a rules-based system that provides a level playing field for everyone…

I want to add a historical remark. Very early forms of human government, such as ancient kingdoms in various parts of the world, had one or a few prominent members of society hold full discretion and decision-making power. Later, we codified rules and decided that even monarchs are not above the law. Endowing the written laws with additional power by making them executable seems like a natural next step.
Post reply on HN