Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do Foundation Model Providers Comply with the EU AI Act?

crfm.stanford.edu

Post reply on HN
Print view View original

31–40 of 75 posts

Re: Do Foundation Model Providers Comply with the EU AI Act?

#31

Post by [deleted] »

Dylan16807 wrote: post #25
hdkrgr wrote: post #20

Earlier quoted context omitted.

I do think this will be a useful metric, and it seems obvious that the hyperscalers will have a feature helping you keep track of energy use and emissions of the resources you rented. But why demand this on the level of an individual model/product? For these foundation models, I think it's reasonable to assume they will all be trained on hyperscaler-provided gpu-clusters, so there'll likely be an off-the-shelf funcit…

Especially for a server GPU, looking up watts and multiplying by time per token should give you a pretty good number.

[deleted]

Re: Do Foundation Model Providers Comply with the EU AI Act?

#32

Post by hdkrgr »

Dylan16807 wrote: post #25
hdkrgr wrote: post #20

Earlier quoted context omitted.

I do think this will be a useful metric, and it seems obvious that the hyperscalers will have a feature helping you keep track of energy use and emissions of the resources you rented. But why demand this on the level of an individual model/product? For these foundation models, I think it's reasonable to assume they will all be trained on hyperscaler-provided gpu-clusters, so there'll likely be an off-the-shelf funcit…

Especially for a server GPU, looking up watts and multiplying by time per token should give you a pretty good number.

Sure... but maybe the GPU is sitting idle 40% of the time while still consuming 200W. Should I have to break this idle energy consumption down onto actual use (assuming the server/gpu is only used for this one model)? I guess it would make sense, but... WHO should do this and then continually update the model documentation when idle rates or the hardware changes?

Re: Do Foundation Model Providers Comply with the EU AI Act?

#33

Post by pbmonster »

bryanrasmussen wrote: post #7

I gotta say I don't see the point of the two compute requirements - the measure energy and reduce energy consumption part seems like it would be something a company would want to do by itself as that is a cost. and I'm not sure what disclosing the training time and power used to train the model tells anyone? on edit: tells anyone that is a consumer of the product and might have legitimate worries which would be a goo…

I think they just learned something from the history of crypto currencies, and this time they want hard data from the beginning. I know it's really apples to oranges, but having this 10 years ago for crypto might have pushed the tech from proof-of-work to e.g. proof-of-stake from the beginning.

And in the end there's no limits in this act. It's not like they're saying "No models above 10 GWh inside the EU!".

Re: Do Foundation Model Providers Comply with the EU AI Act?

#34

Post by gyudin »

mtlmtlmtlmtl wrote: post #28
gyudin wrote: post #10

Earlier quoted context omitted.

350 pages of ramblings that nobody who was voting for this act hasn't actually read?

You are one wicked free thinker dude.

I wish haha, they are just following the guidelines :)) https://www.openculture.com/2022/01/read-the-cias-simple-sab...

Re: Do Foundation Model Providers Comply with the EU AI Act?

#35

Post by londons_explore »

hdkrgr wrote: post #17
csmpltn wrote: post #8

Tracking "Energy" for what is otherwise a compressed version of the entire internet at your disposal seems so green-washed and disingenuous. Whatever "Energy" consumption those models have - it's peanuts compared to the alternatives.

I had the chance to talk to a staffer of one of the MEPs leading the political negotiations in EU parlament committee a few weeks ago. His take was that the pro-tech/pro-business parties conceded the 'AI users must track their energy use'-point to the Greens in the latest draft (which is the parliament's counterproposal to earlier drafts by the commission --think EU executive-- and council --think governments of the…

The problem is, as soon as it's a law, there will be some official way to calculate it, penalties for misreporting, perhaps even a professional who must audit the energy use. Etc. Getting that number gets expensive!

Whereas if it is a company voluntarily reporting it, the number would just be number of GPUs * wattage of GPUs / tokens generated past year = energy per token.

Re: Do Foundation Model Providers Comply with the EU AI Act?

#36

Post by gmm1990 »

csmpltn wrote: post #8

Tracking "Energy" for what is otherwise a compressed version of the entire internet at your disposal seems so green-washed and disingenuous. Whatever "Energy" consumption those models have - it's peanuts compared to the alternatives.

Pretty sure a google search is order/s of magnitude cheaper than a chat gpt one. Training costs may not be comparable. I’m not sure.

Re: Do Foundation Model Providers Comply with the EU AI Act?

#37

Post by nonethewiser »

pbmonster wrote: post #33
bryanrasmussen wrote: post #7

I gotta say I don't see the point of the two compute requirements - the measure energy and reduce energy consumption part seems like it would be something a company would want to do by itself as that is a cost. and I'm not sure what disclosing the training time and power used to train the model tells anyone? on edit: tells anyone that is a consumer of the product and might have legitimate worries which would be a goo…

I think they just learned something from the history of crypto currencies, and this time they want hard data from the beginning. I know it's really apples to oranges, but having this 10 years ago for crypto might have pushed the tech from proof-of-work to e.g. proof-of-stake from the beginning. And in the end there's no limits in this act. It's not like they're saying "No models above 10 GWh inside the EU!".

Why should there be models above 10GWh inside the EU?

Re: Do Foundation Model Providers Comply with the EU AI Act?

#39

Post by gchadwick »

hdkrgr wrote: post #24
gchadwick wrote: post #22

Earlier quoted context omitted.

It's notable that Hugging Face's BLOOM ( https://huggingface.co/bigscience/bloom ) might already be compliant (ignoring the 'member states' requirement which I'm sure they could comply with easily enough, it's about disclosing EU member where the model is on the market, so them simply listing all EU countries in a doc somewhere may suffice). Some may be tempted to point at the lower scores of OpenAI, Google and Faceb…

The full marks for Google in the 'member-states' section made me laugh out loud. -- Bard is not available in the EU. ;)

Good point, so really it should have full marks across the board ;)

Re: Do Foundation Model Providers Comply with the EU AI Act?

#40

Post by Simon321 »

Seems to EU is determined to cripple their AI industry at all costs, we already have so little technology companies...

Foundation models are labelled as 'high risk'!

In my opinion this is way too premature... this would cripple open source AI as well...

> While the act includes open source exceptions for traditional machine learning models, it expressly forbids safe-harbor provisions for open source generative systems.

>Any model made available in the EU, without first passing extensive, and expensive, licensing, would subject companies to massive fines of the greater of €20,000,000 or 4% of worldwide revenue. Opensource developers, and hosting services such as GitHub – as importers – would be liable for making unlicensed models available.

>Open Source LLMs Not Exempt: Open source foundational models are not exempt from the act. The programmers and distributors of the software have legal liability. For other forms of open source AI software, liability shifts to the group employing the software or bringing it to market. (pg 70).

Source: https://technomancers.ai/eu-ai-act-to-target-us-open-source-...

While i'm usually pro-EU they are really overreacting here and the consequences for our economy of crippling a technology with so much potential will be enormous in the long run.

Post reply on HN