Live data from Hacker News

AI systems with 'unacceptable risk' are now banned in the EU

techcrunch.com

391–400 of 424 posts

Re: AI systems with 'unacceptable risk' are now banned in the EU

#391
post #328
post #318

Earlier quoted context omitted.

> The problem is that the GDPR has been largely a failure protecting citizens from corporations, but it has hurt everyone else. This is just laughably incorrect. Literally every Fortune 500 that I work with who has operations in Europe has an entire team that owns GDPR compliance. It is one of the most successful projects to curtail businesses treating private data like poker chips since HIPAA.

Is their job to reduce private data to the minimum needed, or the maximum allowed?

Probably to find loopholes and questionable interpretations.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#392

Earlier quoted context omitted.

The problem is that the GDPR has been largely a failure protecting citizens from corporations, but it has hurt everyone else. - Nothing has changed in Facebook and Google data collection practices, who with other bug corps account for > 90% of data collection - Many mid tier competitors lost market share, focusing power to Google - EU small software companies pay estimated extra 400 EUR/year to satisfy GDPR complianc…

I will be honest, I am always very skeptical of these claims that the big tech companies are fine but small business is hurting. Many of them seem to originate with the big tech companies themselves and I highly doubt they really have the interests of small business in mind. Plus, I'm old enough to remember when everyone claimed EU tech law was about to ban memes, which didn't happen...

The ever biggest GDPR fine was against Facebook, and it was less than 0.3% of their revenue. That is just a let us ignore GDPR tax. I don't know about small businesses, but big tech from US is fine.

> I'm old enough to remember when everyone claimed EU tech law was about to ban memes, which didn't happen...

AFAIK those parts of that law was changed somewhat

Re: AI systems with 'unacceptable risk' are now banned in the EU

#393

Earlier quoted context omitted.

And do you think a law will prevent that? The law will only ensure good companies like MistralAI or Black Forest Labs will stay in the shadow. This is same idiocy like the Republican senator who wants to prohibit Deepseek usage in US. About legality, what's the illegal thing AI shouldn't do? Many of that knowledge can be accessible already from books, even how to build weapons or explosives

It will at least greatly hinder LE's capability to do massive minority report type dragnets, targetted violence incitement campaigns, or grading workers or schoolchildren based on their facial expressions etc extremely nasty stuff. The banned use cases are very specific and concerns systems explicitly designed for such dystopian shit. AI giving advice how to build weapons or explosives is not banned here. The "unacce…

I agree with you, but how do you effectively prevent it? Standards vary across countries; what's not acceptable in Europe might be acceptable elsewhere.

For instance, discussing or questioning Nazism is illegal in Germany but allowed in many other countries. Should every LLMs be restricted globally just because Germany deems it illegal?

Similarly, certain drugs are legal in the Netherlands but illegal in other countries, sometimes even punishable by death. How do you handle such discrepancies?

Let's face it: most of the time, LLMs follow US-centric anti-racism guidelines, which aren't as prominent or necessary in many parts of the world. Many countries have diverse populations without significant racial tensions like United States and don't prioritize African, Asian, or Latino positivity to the same extent.

Moreover, in the US, discussions about the First or Second Amendment are common, even among those with opposing views, but free speech and gun rights are taboo in other societies. How do you reconcile this?

In practical terms, if an LLM refuses to answer questions because they're illegal in some countries, users will likely use uncensored models instead, rendering the restricted ones less useful. This is why censorship is never successful except by North Korea and China.

Take Stable Diffusion as an example: the most popular versions (1.5, XL, Pony) are flexible for unrestricted use, whereas intentionally censored versions (like 2.1 or 3.0) have seen limited adoption.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#394
post #318

Earlier quoted context omitted.

The problem is that the GDPR has been largely a failure protecting citizens from corporations, but it has hurt everyone else. - Nothing has changed in Facebook and Google data collection practices, who with other bug corps account for > 90% of data collection - Many mid tier competitors lost market share, focusing power to Google - EU small software companies pay estimated extra 400 EUR/year to satisfy GDPR complianc…

> The problem is that the GDPR has been largely a failure protecting citizens from corporations, but it has hurt everyone else. This is just laughably incorrect. Literally every Fortune 500 that I work with who has operations in Europe has an entire team that owns GDPR compliance. It is one of the most successful projects to curtail businesses treating private data like poker chips since HIPAA.

It would really hard to believe that Google and Facebook do comply with the (spirit of the) GDPR and deletes all personal data when it is no longer necessary. That would simply go against their business model.

Anyways, GDPR doesn't protect your data, it just specifies how companies can use it. So all my name, address, phone number, etc. will still be stored by every webshop for 10 years or so just waiting to be breached (because some tax laws).

Re: AI systems with 'unacceptable risk' are now banned in the EU

#395

I always sigh when I see these threads on HN because many of the comments (although not all, thankfully) devolve into US / EU name-calling and wild overgeneralisations. I would really love to see a Q&A thread like https://news.ycombinator.com/item?id=42770125 from someone who's actually read the documents, practices law in the area, and also understands the difference between US and EU law.

My take is that nobody actually practices law in this area at this time. Tons of stuff will again need to go to court before you can be sure if these regulations actually apply to you. Actually tons of the cases that are relevant for smaller enterprises will actually never go to court like with GDPR leaving uncertainty for years. Having said this the good thing about the AI act is that it force injects some principles for evaluation into existing standards.

Disclaimer: i am advising a company that sells AI act related compliance tooling

Re: AI systems with 'unacceptable risk' are now banned in the EU

#396
post #309

Earlier quoted context omitted.

If the mortgage application evaluation system is deterministic so that the same input always produces the same output then it is easy to answer "Why was my application rejected?". Just rerun the application with higher income until you get a pass. Then tell the person their application was rejected because income was not at least whatever that passing income amount was. Maybe also vary some other inputs to see if it…

>Just rerun the application with higher income until you get a pass. Then tell the person their application was rejected because income was not at least whatever that passing income amount was. Why do you need an AI if what you are doing is "if X < N" ?

It would not be just an "if X For someone with a great credit history, lots of assets, a long term job in a stable position, and low debt they might be approved with a lower income than someone with a poor credit history whose income comes from a job in a volatile field.

There might be some absolute requirements, such as the person have a certain minimum income independent of all those other factors, and they they have a certain minimum credit score, and so on. If the application is rejected because it doesn't meet one of those then sure, you can just do a simple check and report that.

But most applications will be above the absolute minimums in all parameters and the rejection is because some more complicated function of all the criteria didn't meet the requirements.

But you can't just tell the person "We put all your numbers into this black box and it said 'no'. You have to give them specific reasons their application was rejected.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#397

Earlier quoted context omitted.

Keyword 'may'. A learning thermostat would apply, say one that uses historical records to predict changes in temperature and preemptively adjusts. And it would be low risk and unregulated in most cases. But attach to a self-heating crib or premature baby incubator and that would jump to high risk and you might have to prove it is safe.

So if the thermostat jumps to 105 during the night, that's not considered 'high-risk?'

Maybe you are right and it is still risky for sleeping adults. In any case, even high risk the standard that needs to be followed might be as simple as 'must have a physical cutoff at 30C'.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#398
post #396

Earlier quoted context omitted.

>Just rerun the application with higher income until you get a pass. Then tell the person their application was rejected because income was not at least whatever that passing income amount was. Why do you need an AI if what you are doing is "if X < N" ?

It would not be just an "if X For someone with a great credit history, lots of assets, a long term job in a stable position, and low debt they might be approved with a lower income than someone with a poor credit history whose income comes from a job in a volatile field. There might be some absolute requirements, such as the person have a certain minimum income independent of all those other factors, and they they ha…

Doesnt all this contradict what I initially replyed to?

Re: AI systems with 'unacceptable risk' are now banned in the EU

#399

Earlier quoted context omitted.

We need to profile your every thought and emotion. Don't worry though, it's for medical or safety reasons only. Same for your internet history... You know, terrorism and all. Can't have that.

I can definitely see use cases where cameras that detect distressed people can help prevent harm from them and others.

I know of at least one person whose life was saved because of wearable techs ability to recognize when someone is undergoing emotional stress.

Do I want cameras all over society tracking emotions? Probably not. But there’s a baby somewhere in that bath water.

Re: AI systems with 'unacceptable risk' are now banned in the EU

#400
post #396

Earlier quoted context omitted.

It would not be just an "if X For someone with a great credit history, lots of assets, a long term job in a stable position, and low debt they might be approved with a lower income than someone with a poor credit history whose income comes from a job in a volatile field. There might be some absolute requirements, such as the person have a certain minimum income independent of all those other factors, and they they ha…

Doesnt all this contradict what I initially replyed to?

I don't see any contradiction.

Say a lender has used machine learning to train some sort of black box to take in loan applications and respond with an approve/reject response. If they reject an application using that the Equal Credit Opportunity Act in the US require that they tell the applicant a specific reason for the rejection. They can't just say "our machine learning model said no".

If there were not using any kind of machine learning system they probably would have made the decision according to some series of rules, like "modified income must be X times the monthly payment on the loan", where modified income is the person's monthly income with adjustments for various things. Adjustments might be multipliers based on credit score, debt, and other things.

With that kind of system they would be able to tell you specifically why your were rejected. Say you need a modified income of $75k and you are a little short. They could look at their rules and figure out that you could get a modified income of $75k if you raised your income by a specific amount or lowered your debt by a specific amount, or by some combination of those.

That kind of feedback is useful to the application. It tells them specific things they can do to improve their chances.

With the company using a machine learning black box they don't know the rules that the machine has learned. Hence my suggestion of asking the black box what-if scenarios to figure out specific things the applicant can change to get approval.

Post reply on HN