Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI engineers claim new algorithm reduces AI power consumption by 95%

tomshardware.com

Post reply on HN
Print view View original

71–80 of 174 posts

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#71

Post by andrewstuart »

jdiez17 wrote: post #69
andrewstuart wrote: post #67

Earlier quoted context omitted.

https://github.com/microsoft/BitNet "The first release of bitnet.cpp is to support inference on CPUs. bitnet.cpp achieves speedups of 1.37x to 5.07x on ARM CPUs, with larger models experiencing greater performance gains. Additionally, it reduces energy consumption by 55.4% to 70.0%, further boosting overall efficiency. On x86 CPUs, speedups range from 2.37x to 6.17x with energy reductions between 71.9% to 82.2%. Furt…

Damn. Seems almost too good to be true. Let’s see where this goes in two weeks.

Intel and AMD will be extremely happy.

Nvidia will be very unhappy.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#72

Post by jimmaswell »

rossjudson wrote: post #70
jimmaswell wrote: post #42

Earlier quoted context omitted.

Faster progress was absolutely worth it. Spending years agonizing over theory to save a bit of electric would have been a massive disservice to the world.

You're sort of presuming that LLMs are going to be a massive service to the world there, aren't you? I think the jury is still out on that one.

They already have been. Even just in programming, even just Copilot has been a life changing productivity booster.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#73

Post by rob_c »

manquer wrote: post #22
kayo_20211030 wrote: post #10

Extraordinary claims require extraordinary evidence. Maybe it's possible, but consider that some really smart people, in many different groups, have been working diligently in this space for quite a while; so claims of 95% savings on energy costs _with equivalent performance_ is in the extraordinary category. Of course, we'll see when the tide goes out.

It is a click bait headline the claim itself is not extraordinary. the preprint from arxiv was posted here some time back . The 95% gains is specifically only for multiplication operations, inference is compute light and memory heavy in the first place so the actual gains would be far less smaller . Tech journalism (all journalism really) can hardly be trusted to publish grounded news with the focus on clicks and rev…

Bingo,

We have a winner. Glad that came from someone not in my lectures on ML network design

Honestly, thanks for beeting me to this comment

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#74

Post by keybored »

TheRealPomax wrote: post #56
didgetmaster wrote: post #51

Maybe I am just a natural skeptic, but whenever I see a headline that says 'method x reduces y by z%'; but when you read the text it instead says that optimizing some step 'could potentially reduce y by up to z%'; I am suspicious. Why not publish some actual benchmarks that prove your claim in even a few special cases?

Because as disappointing as modern life is, you need clickbait headlines to drive traffic. You did the right thing by reading the article though, that's where the information is, not the title.

Headlines: what can they do, they need that for the traffic

Reader: do the moral thing and read the article, not just the title

How is that balanced.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#75

Post by mattxxx »

GistNoesis wrote: post #36

Does https://en.wikipedia.org/wiki/Jevons_paradox apply in this case ?

That's interesting.

Obviously, energy cost creates a barrier to entry, so reduction of cost reduces the barrier to entry... which adds more players... which increases demand.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#76

Post by wruza »

jimmaswell wrote: post #72
rossjudson wrote: post #70

Earlier quoted context omitted.

You're sort of presuming that LLMs are going to be a massive service to the world there, aren't you? I think the jury is still out on that one.

They already have been. Even just in programming, even just Copilot has been a life changing productivity booster.

Are you sure it’s a life changing productivity booster? Sometimes I look at my projects and wonder how would I explain it to an LLM what this code should have done if it didn’t exist yet. Must be a shitton of boilerplate programming for copilot to be a life-changing experience.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#77

Post by etcd »

djoldman wrote: post #7

https://arxiv.org/abs/2410.00907 ABSTRACT Large neural networks spend most computation on floating point tensor multiplications. In this work, we find that a floating point multiplier can be approximated by one integer adder with high precision. We propose the linear-complexity multiplication (L-Mul) algorithm that approximates floating point number multiplication with integer addition operations. The new algorithm c…

I feel like I have seen this idea a few times but don't recall where but stuff posted via HN.

Here https://news.ycombinator.com/item?id=41784591 but even before that. It is possibly one of those obvious ideas to people steeped in this.

To me intuitively using floats to make ultimatelty boolean like decisions seems wasteful but that seemed like the way it had to be to have diffetentiable algorithms.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#78

Post by Scene_Cast2 »

fuzzfactor wrote: post #34
crazygringo wrote: post #20

Earlier quoted context omitted.

No. But it does potentially mean that either current or future-tweaked GPUs could run a lot more efficiently -- meaning much faster or with much less energy consumption. You still need the GPU parallelism though.

I had a feeling it had to be something like massive waste due to a misguided feature of the algorithms that shouldn't have been there in the first place. Once the "math is done" quite likely it would have paid off better than most investments for the top people to have spent a few short years working with grossly underpowered hardware until they could come up with amazing results there before scaling up. Rather than…

This is a bit like recommending to skip vacuum tubes, think hard and invent transistors.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#79

Post by etcd »

baq wrote: post #58
didgetmaster wrote: post #51

Maybe I am just a natural skeptic, but whenever I see a headline that says 'method x reduces y by z%'; but when you read the text it instead says that optimizing some step 'could potentially reduce y by up to z%'; I am suspicious. Why not publish some actual benchmarks that prove your claim in even a few special cases?

OTOH you have a living proof that an amazingly huge neural network can work on 20W of power, so expecting multiple orders of magnitude in power consumption reduction is not unreasonable.

Mitochondria are all you need.

Should be able to go more efficient as the brain has other constraints such as working at 36.7 degrees C etc.

Post reply on HN