Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI systems with 'unacceptable risk' are now banned in the EU

techcrunch.com

Post reply on HN
Print view View original

231–240 of 424 posts

Re: AI systems with 'unacceptable risk' are now banned in the EU

#231

Post by HenryBemis »

DoingIsLearning wrote: post #33

I am not expert but there seems to be an overlap in the article between 'AI' and well ... just software, or signal processing: - AI that collects “real time” biometric data in public places for the purposes of law enforcement. - AI that creates — or expands — facial recognition databases by scraping images online or from security cameras. - AI that uses biometrics to infer a person’s characteristics - AI that collect…

I've worked with the bureaucrats in Brussels on tech/privacy topics.

Their deep meaning is "we don't want machines to make decisions". A key point for them has always been "explainability".

GDPR has a provision about "profiling" and "automated decision making" for key aspects of life. E.g. if you ask for a mortgage (pretty important life changing/affecting decision) and the bank rejects it you a) can ask them "why" and they MUST explain, in writing, and b) if the decision was made in a system that was fed your data (demographic & financial) you can request that a Human to repeat the 'calculations'.

Good luck having ChatGTP explaining.

They are trying to avoid having the dystopian nightmare of the (apologies - I don't mean to disrespect the dead, I mean to disrespect the industry) Insurance & Healthcare in the US, where a system gets to decide 'your claim is denied' against humans' (doctors in this case)(sometimes imperfect) consultations because one parameter writes "make X amount of profit above all else" (perhaps not coded with this precise parameter but somehow else).

Now, understanding the (personal) data collection and send to companies in the US (or other countries) that don't fall under the Adequacy Decisions [0] and combining that with the aforementioned (decision-making) risks, using LLMs in Production is 'very risky'.

Using Copilot for writing code is very much different because there the control of "converting the code to binaries, and moving said binaries to Prod env." (they used to call them Librarians back in the day...), so Human Intervention is required to do code review, code test, etc (just in case SkyNet wrote code to export the data 'back home' to OpenAI, xAI, or any other AI company it came from).

I haven't read the regulation lately/in its final text (I contributed and commented some when it was still being drafted), and/but I remember the discussions on the matter.

[0]: https://commission.europa.eu/law/law-topic/data-protection/i...

EDIT: ultimately we want humans to have the final word, not machines.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#232

Post by hkwerf »

rustc wrote: post #22

Does this affect open weights AI releases? Or is the ban only on the actual use for the listed cases? Because you can use open weights Mistral models to implement probably everything on that list.

The law specifically refers to putting a product in to service or placing a product on the market if they are on the prohibited (article 5) or high-risk (article 6, annex III) list and don't have an applicable exception. IANAL, but it's pretty obvious that a set of weights alone clearly is not such a product.

Also note that the law has explicit exceptions for research, development, open source and personal use.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#233

Post by close04 »

yorwba wrote: post #214
maxrmk wrote: post #185

Earlier quoted context omitted.

I'm not sure what they intended this to apply to. LLM based systems don't change their own operation (at least, not more so than anything with a database). We'll probably have to wait until they fine someone a zillion dollars to figure out what they actually meant.

For LLMs we have "for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments".

For either option you can trace the intention of the definitions to "was it a human coding the decision or not". Did a human decide the branches of the literal or figurative "if"?

The distinction is accountability. Determining whether a human decided the outcome, or it was decided by an obscure black box where data is algebraically twisted and turned in a way no human can fully predict today.

Legally that accountability makes all the difference. It's why companies scurry to use AI for all the crap they want to wash their hands of. "Unacceptable risk AI" will probably simply mean "AI where no human accepted the risk", and with it the legal repercussions for the AI's output.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#235

Post by oneeyedpigeon »

philipov wrote: post #183
hotstickyballs wrote: post #165

Earlier quoted context omitted.

It doesn't seem to be clear to me whether auto-formatted code (or even generated code from copilot for example) would be classified as AI.

It seems to me the key phrase in that definition is "that may exhibit adaptiveness after deployment" - If your code doesn't change its own operation without needing to be redeployed, it's not AI under this definition. If adaptation requires deployment, such as pushing a new version, that's not AI.

It's unclear whether "may" means "can (and does)" or whether it renders that entire clause optional.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#236

Post by hkwerf »

johndhi wrote: post #134
Havoc wrote: post #120

For once that doesn’t seem overly broad. Pretty much agree with all of the list

The "high risk" list is where the breadth comes in

The "high risk" list, though, is essentially traditional safety functions (article 6) and functions that affect fundamental rights and access to basic services (annex III)? It's not that broad at all either.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#237

Post by oneeyedpigeon »

impossiblefork wrote: post #81
DoingIsLearning wrote: post #33

I am not expert but there seems to be an overlap in the article between 'AI' and well ... just software, or signal processing: - AI that collects “real time” biometric data in public places for the purposes of law enforcement. - AI that creates — or expands — facial recognition databases by scraping images online or from security cameras. - AI that uses biometrics to infer a person’s characteristics - AI that collect…

I wouldn't be surprised if it does cover all software. After all, chess solvers are AI.

Chess solvers are more AI than 90% of the things currently being touted as AI!

Re: AI systems with 'unacceptable risk' are now banned in the EU

#238

Post by sofixa »

cabalamat wrote: post #217
spacemanspiff01 wrote: post #47

Earlier quoted context omitted.

From the laws text: For the purposes of this Regulation, the following definitions apply: (1) ‘AI system’ means a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that c…

> and that may exhibit adaptiveness after deployment So if an AI can't change its weights after deployment, it's not really an AI? That doesn't make sense. As for the other criteria, they're so vague I think a thermostat might apply.

> As for the other criteria, they're so vague I think a thermostat might apply.

As long as the thermostat doesn't control people's lives, that's fine.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#239

Post by oneeyedpigeon »

caseyy wrote: post #200
jeffdotdev wrote: post #94

Earlier quoted context omitted.

There is no law that isn't open to interpretation. There is a reason for the judicial branch of government.

Well, the laws in civil law countries that practice legal literalism are not open to interpretation. Eastern Europe, much of which is a part of the EU, is quite literalist. The understanding is that interpreting laws leads to bias, partiality, and injustice; while following the letter of the law equally in each situation is the most just approach.

I don't believe that's even possible — I'd love to see an example. How do you define anything 100% literally, 100% unambiguously? You'd have to include the entire language in your definition for a start, and keep that constantly updated.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#240

Post by oneeyedpigeon »

DocTomoe wrote: post #123
jeffgreco wrote: post #98

Earlier quoted context omitted.

Yet better privacy protections than we in the States enjoy.

The EU in a nutshell: You can't have nice things, but on the bright side Google/Apple/Facebook won't know what you had for dinner. Now give us your whole financial transaction and travel history, so we can share it with the US, a hostile country, citizen!

The USA in a nutshell:

We elected a President who tried to lead an armed insurrection but we'll never press criminal charges because we elected him President again.

Sorry, but anything the EU has ever done pales in comparison with that.

Post reply on HN