Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI systems with 'unacceptable risk' are now banned in the EU

techcrunch.com

Post reply on HN
Print view View original

411–420 of 424 posts

Re: AI systems with 'unacceptable risk' are now banned in the EU

#411

Post by justonceokay »

Nasrudith wrote: post #404
justonceokay wrote: post #349

Earlier quoted context omitted.

I think in the medium-term future we will have our own real-life Butlerian Jihad against thinking machines. Maybe they won’t be outright banned, but there needs to be some conservative force that pushes back against progress for the sake of “because we can”. Is this legislation premature? Maybe. But I don’t think it will be the last or the the most comprehensive within our lifetimes.

Why the fuck do people cite the Butlerian jihad as a good thing instead of what it is: stupidity writ large? "There must always be a lich king" logic in play. No, there really fucking doesn't.

Who is the lich king? I’m not following

Re: AI systems with 'unacceptable risk' are now banned in the EU

#412

Post by fcantournet »

johndhi wrote: post #133
abdullahkhalids wrote: post #113

Earlier quoted context omitted.

Seems very reasonable. Not all software has the same risk profile, and autonomous+adaptive software certainly have a more dangerous profile than simpler software, and should be regulated differently.

What? Why? Shouldn't those same use cases all be banned regardless of what tech is used to build them?

I mean on the one hand I agree, none of these use cases seem legitimate, and most give off very totalitarian vibes.

However for those that might not be purely 1984 inspired, I do think that we need to have legislation that is capable of making the distinction between : - algorithms that can be reasoned about and analysed - "AI" systems that resist such analysis

The main issue is around responsability. Who would be held responsible for illegal (discriminatory) biases in an AI systems ? How would regulators even detect, specify and quantify those biases ?

In non-AI systems, we can analyse the algorithm and evaluate if the biases are due to errors (negligence) or are by design (malice / large scale criminality)

Re: AI systems with 'unacceptable risk' are now banned in the EU

#413

Post by Muromec »

tzs wrote: post #400
Muromec wrote: post #398

Earlier quoted context omitted.

Doesnt all this contradict what I initially replyed to?

I don't see any contradiction. Say a lender has used machine learning to train some sort of black box to take in loan applications and respond with an approve/reject response. If they reject an application using that the Equal Credit Opportunity Act in the US require that they tell the applicant a specific reason for the rejection. They can't just say "our machine learning model said no". If there were not using any…

>That kind of feedback is useful to the application. It tells them specific things they can do to improve their chances.

In that sense it's very practical, but it kicks the can down the road. Maybe the thing has a hidden parameter that represents the risk of the applicant being fired, which increases the threshold by 5% if the salary is a round number. Or it is more likely to reject everyone with an income between 73 and 75k because it learned this is a proxy to a parameter you are explicitly forbidden to have.

Let's just say it doesn't have a discontinuity, and actually produces the threshold which is deterministically compared with your income. How does it come up with this threshold? You may not be required to disclosed that to the applicant, but it would be a shame if people will figure out that the threshold is consistently higher for a certain population (for example people who's given name ends with a vowel).

It's fairly reasonable to for a regulator to ask you to demonstrate it doesn't do any of this things.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#414

Post by jampekka »

waltercool wrote: post #393
jampekka wrote: post #205

Earlier quoted context omitted.

It will at least greatly hinder LE's capability to do massive minority report type dragnets, targetted violence incitement campaigns, or grading workers or schoolchildren based on their facial expressions etc extremely nasty stuff. The banned use cases are very specific and concerns systems explicitly designed for such dystopian shit. AI giving advice how to build weapons or explosives is not banned here. The "unacce…

I agree with you, but how do you effectively prevent it? Standards vary across countries; what's not acceptable in Europe might be acceptable elsewhere. For instance, discussing or questioning Nazism is illegal in Germany but allowed in many other countries. Should every LLMs be restricted globally just because Germany deems it illegal? Similarly, certain drugs are legal in the Netherlands but illegal in other countr…

The "unacceptable risk" category is not about chatbots or LLM capabilities in general. It's not about censorship or banned topics. It's about certain specific applications and systems tailored for them.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#415

Post by everforward »

fxtentacle wrote: post #338
miohtama wrote: post #315

Earlier quoted context omitted.

The problem is that the GDPR has been largely a failure protecting citizens from corporations, but it has hurt everyone else. - Nothing has changed in Facebook and Google data collection practices, who with other bug corps account for > 90% of data collection - Many mid tier competitors lost market share, focusing power to Google - EU small software companies pay estimated extra 400 EUR/year to satisfy GDPR complianc…

"Nothing has changed in Facebook and Google data collection practices" https://noyb.eu/en Facebook and Google got sued, paid fines, and changed their behavior. I can do an easy export of all of my FB and G data, thanks to the GDPR. "EU small software companies pay estimated extra 400 EUR/year to satisfy GDPR compliance" WTF? no! I work with several small companies and it's super easy to just NOT store anyone's birthd…

400 Eur is pretty small, it rings true to me. Maybe not in literal costs, but 400 Euro of employee salaries is pretty low. Figuring out how to not store IPs but also be able to block malicious IPs probably costs at least 400 Euro in employee salaries.

At 40k EUR / year in salary, that's about 1.6 hours a month dealing with GDPR. That sounds about right; it's like 5 hours a quarter deploying anonymizers or updating code to export the data you have on people. I honestly expected it to be higher; I would have thought it was in the realm of 40 hours a quarter just doing mundane things. Auditing to make sure PII didn't sneak in somewhere, updating anonymizer code/deployments and reviewing the same.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#416

Post by xp84 »

verzali wrote: post #317
miohtama wrote: post #315

Earlier quoted context omitted.

The problem is that the GDPR has been largely a failure protecting citizens from corporations, but it has hurt everyone else. - Nothing has changed in Facebook and Google data collection practices, who with other bug corps account for > 90% of data collection - Many mid tier competitors lost market share, focusing power to Google - EU small software companies pay estimated extra 400 EUR/year to satisfy GDPR complianc…

I will be honest, I am always very skeptical of these claims that the big tech companies are fine but small business is hurting. Many of them seem to originate with the big tech companies themselves and I highly doubt they really have the interests of small business in mind. Plus, I'm old enough to remember when everyone claimed EU tech law was about to ban memes, which didn't happen...

You can be skeptical but I’ve worked at multiple small businesses since GDPR and CCPA came to be, and each of them has zero interest in “selling your data” - everyone just wants to run ads and track which ones work. And yet complying with GDPR has been onerous and costly in every one of them. And did nothing to benefit our customers or website visitors. The only winners are the lawyers and firms that specialize in selling “compliance as a service” basically.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#417

Post by martin_a »

cmenge wrote: post #407
martin_a wrote: post #314

Earlier quoted context omitted.

That's not true. IP addresses might be processed in regards to article 6.1 c) or 6.1 f) but only for these very narrowly defined use cases and in accordance with article 5. So, purge your logs after 14/30 days and don't use the ip address for anything else and you will be fine.

Where do the 14/30 days come from?

Gut feeling (kind of).

There are rulings that access providers are/were allowed to save full IP addresses for up to 7 days to handle misuse of services etc. and any longer storage seems unnecessary and unlawful.

In other cases there were recommendations of up to 30 days, ideally with anonymized addresses where the last one or two triplets are automatically being removed. I've also seen 30 days as kind of the default setting for automatic log purging with shared webhosters.

Our lawyer told us that he estimates that saving full IP addresses for 14 days in logfiles would be fine in regards of preventing/tracking misuse of services or attacks against the infrastructure.

If this would ever come to court it would most probably be up to the judge to see whether this is really fine or already too much. Therefore we had to document the process and why we think 14 days is reasonable and so on.

The GDPR lacks a specific time frame and I think that's okay. There's always some "wiggle room" in European laws, it's about not misusing that room and sincerely acting in the best interest of everybody.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#418

Post by johndhi »

hkwerf wrote: post #236
johndhi wrote: post #134

Earlier quoted context omitted.

The "high risk" list is where the breadth comes in

The "high risk" list, though, is essentially traditional safety functions (article 6) and functions that affect fundamental rights and access to basic services (annex III)? It's not that broad at all either.

"fundamental rights" is a very broad concept in eu.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#419

Post by anothernewdude »

Muromec wrote: post #270
anothernewdude wrote: post #182

Earlier quoted context omitted.

> Notably, Recital 12 says the definition "should not cover systems that are based on the rules defined solely by natural persons to automatically execute operations." That's every AI system. It follows the rules defined solely by the programmers (who I suppose might sometimes stretch the definition of natural persons) who made pytorch or whatever framework.

If the thinking machine rejects my mortgage application, it should be possible to point out which exact rule triggered the rejection. With rules explicitly set by an operator it's possible. It's also possible to say that the rules in place comply with the law and stay compliant during the operation, for example it doesn't unintentionally guess that I'm having another citizenship based on my surname or postal code.

Sure, it's the rule that multiplies all the weights of matrix C1, with your transformed inputs. It's right there. What part of that rule don't you understand?

Re: AI systems with 'unacceptable risk' are now banned in the EU

#420

Post by Zenst »

pjc50 wrote: post #348
Zenst wrote: post #96

Earlier quoted context omitted.

I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front. Maybe that will get challenged as disability discrimination by some Autistic group. Which would be interesting. As with most things, there are rules, and exceptions to those rules - no shoe fits everyone, though forcing people to wear the wrong shoe size, c…

I can definitely find you autistic people who would hate having such a device pointed at them, because they don't mask the ""correct"" emotional state well enough.

Was on about Autistic people being able to use it to understand the other person, not the other way around, which would be a nightmare for them, which sadly is what they get in real life anyhow already. Though might be useful to warn them that they are sending the wrong signals, so a mixed bag on that way around.
Post reply on HN