Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI engineers claim new algorithm reduces AI power consumption by 95%

tomshardware.com

Post reply on HN
Print view View original

41–50 of 174 posts

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#41

Post by fidotron »

robomartin wrote: post #9

I posted this about a week ago: https://news.ycombinator.com/item?id=41816598 This has been done for decades in digital circuits, FPGA’s, Digital Signal Processing, etc. Floating point is both resource and power intensive and using FP without the use of dedicated FP processing hardware is something that has been avoided and done without for decades unless absolutely necessary.

Right, the ML people are learning, slowly, about the importance of optimizing for silicon simplicity, not just reduction of symbols in linear algebra.

Their rediscovery of fixed point was bad enough but the “omg if we represent poses as quaternions everything works better” makes any game engine dev for the last 30 years explode.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#42

Post by jimmaswell »

fuzzfactor wrote: post #34
crazygringo wrote: post #20

Earlier quoted context omitted.

No. But it does potentially mean that either current or future-tweaked GPUs could run a lot more efficiently -- meaning much faster or with much less energy consumption. You still need the GPU parallelism though.

I had a feeling it had to be something like massive waste due to a misguided feature of the algorithms that shouldn't have been there in the first place. Once the "math is done" quite likely it would have paid off better than most investments for the top people to have spent a few short years working with grossly underpowered hardware until they could come up with amazing results there before scaling up. Rather than…

Faster progress was absolutely worth it. Spending years agonizing over theory to save a bit of electric would have been a massive disservice to the world.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#43

Post by Forgeties79 »

jimmaswell wrote: post #42
fuzzfactor wrote: post #34

Earlier quoted context omitted.

I had a feeling it had to be something like massive waste due to a misguided feature of the algorithms that shouldn't have been there in the first place. Once the "math is done" quite likely it would have paid off better than most investments for the top people to have spent a few short years working with grossly underpowered hardware until they could come up with amazing results there before scaling up. Rather than…

Faster progress was absolutely worth it. Spending years agonizing over theory to save a bit of electric would have been a massive disservice to the world.

“A bit”?

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#44

Post by quantadev »

I wonder if someone has feed this entire "problem" into the latest Chat GPT-01 (the new model with reasoning capability), and just fed it in all the code for a Multilayer Perceptron and then given it the task/prompt of finding ways to implement the same network using only integer operations.

Surely even the OpenAI devs must have done this like the minute they got done training that model, right? I wonder if they'd even admit it was an AI that came up with the solution rather than just publishing it, and taking credit. haha.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#45

Post by kayo_20211030 »

Randor wrote: post #37
kayo_20211030 wrote: post #30

Earlier quoted context omitted.

I'm not an AI person, in any technical sense. The savings being claimed, and I assume verified, are on ARM and x86 chips. The piece doesn't mention swapping mult to add, and a 1-bit LLM is, well, a 1-bit LLM. Also, > Additionally, it reduces energy consumption by 55.4% to 70.0% With humility, I don't know what that means. It seems like some dubious math with percentages.

> I don't know what that means. It seems like some dubious math with percentages. I would start by downloading a 1.58 model such as: https://huggingface.co/HF1BitLLM/Llama3-8B-1.58-100B-tokens Run the non-quantized version of the model on your 3090/4090 gpu and observe the power draw. Then load the 1.58 model and observe the power usage. Sure, the numbers have a wide range because there are many gpu/npu to make the c…

Good one!

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#46

Post by kayo_20211030 »

stefan_ wrote: post #39
kayo_20211030 wrote: post #10

Extraordinary claims require extraordinary evidence. Maybe it's possible, but consider that some really smart people, in many different groups, have been working diligently in this space for quite a while; so claims of 95% savings on energy costs _with equivalent performance_ is in the extraordinary category. Of course, we'll see when the tide goes out.

I mean, all these smart people would rather pay NVIDIA all their money than make AMD viable. And yet they tell us its all MatMul.

Both companies are doing pretty well. Why don't you think AMD is viable?

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#48

Post by Dylan16807 »

andrewstuart wrote: post #15

The ultimate “you’re doing it wrong”. For he sake of the climate and environment it would be nice to be true. Bad news for Nvidia. “Sell your stock” bad. Does it come with a demonstration?

Bad news for Nvidia how? Even ignoring that the power savings are only on one type of instruction, 20x less power doesn't mean it runs 20x faster. You still need big fat GPUs.

If this increases integer demand and decreases floating point demand, that moderately changes future product design and doesn't do much else.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#49

Post by sroussey »

kayo_20211030 wrote: post #30
Randor wrote: post #19

Earlier quoted context omitted.

The energy claims up to ~70% can be verified. The inference implementation is here: https://github.com/microsoft/BitNet

I'm not an AI person, in any technical sense. The savings being claimed, and I assume verified, are on ARM and x86 chips. The piece doesn't mention swapping mult to add, and a 1-bit LLM is, well, a 1-bit LLM. Also, > Additionally, it reduces energy consumption by 55.4% to 70.0% With humility, I don't know what that means. It seems like some dubious math with percentages.

Not every instruction on a CPU or GPU uses the same amount of power. So if you could rewrite your algorithm to use more power efficient instructions (even if you technically use more of them), you can save overall power draw.

That said, time to market has been more important than any cares of efficiency for some time. Now and in the future, there is more of a focus on it as the expenses in equipment and power have really grown.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#50

Post by pcl »

fuzzfactor wrote: post #34
crazygringo wrote: post #20

Earlier quoted context omitted.

No. But it does potentially mean that either current or future-tweaked GPUs could run a lot more efficiently -- meaning much faster or with much less energy consumption. You still need the GPU parallelism though.

I had a feeling it had to be something like massive waste due to a misguided feature of the algorithms that shouldn't have been there in the first place. Once the "math is done" quite likely it would have paid off better than most investments for the top people to have spent a few short years working with grossly underpowered hardware until they could come up with amazing results there before scaling up. Rather than…

Isn’t this paper pretty much about spending a few short years to improve the performance? Or are you arguing that the same people who made breakthroughs over the last few years should have also done the optimization work?
Post reply on HN