[deleted]
AI engineers claim new algorithm reduces AI power consumption by 95%
21–30 of 174 posts
Re: AI engineers claim new algorithm reduces AI power consumption by 95%
#22Extraordinary claims require extraordinary evidence. Maybe it's possible, but consider that some really smart people, in many different groups, have been working diligently in this space for quite a while; so claims of 95% savings on energy costs _with equivalent performance_ is in the extraordinary category. Of course, we'll see when the tide goes out.
It is a click bait headline the claim itself is not extraordinary.
the preprint from arxiv was posted here some time back .
The 95% gains is specifically only for multiplication operations, inference is compute light and memory heavy in the first place so the actual gains would be far less smaller .
Tech journalism (all journalism really) can hardly be trusted to publish grounded news with the focus on clicks and revenue they need to survive.
Re: AI engineers claim new algorithm reduces AI power consumption by 95%
#23Isn't this just taking advantage of "log(x) + log(y) = log(xy)"? The IEEE754 floating-point representation stores floats as sign, mantissa, and exponent -- ignore the first two (you quantitized anyway, right?), and the exponent is just an integer storing log() of the float.
Not quite: It's taking advantage of (1+a)(1+b) = 1 + a + b + ab. And where a and b are both small-ish, ab is really small and can just be ignored. So it turns the (1+a)(1+b) into 1+a+b. Which is definitely not the same! But it turns out, machine guessing apparently doesn't care much about the difference.
You might then as well replace the multiplication by the addition in the original network. In that case you're not even approximating anything.
Am I missing something?
Re: AI engineers claim new algorithm reduces AI power consumption by 95%
#24Duplicate, posted on October 9: https://news.ycombinator.com/item?id=41784591
Re: AI engineers claim new algorithm reduces AI power consumption by 95%
#25Extraordinary claims require extraordinary evidence. Maybe it's possible, but consider that some really smart people, in many different groups, have been working diligently in this space for quite a while; so claims of 95% savings on energy costs _with equivalent performance_ is in the extraordinary category. Of course, we'll see when the tide goes out.
re: all above/below comments. It's still an extraordinary claim.
I'm not claiming it's not possible, nor am I claiming that it's not true, or, at least, honest.
But, there will need to be evidence that using real machines, and using real energy an _equivalent performance_ is achievable. A defense that "there are no suitable chips" is a bit disingenuous. If the 95% savings actually has legs some smart chip manufacturer will do the math and make the chips. If it's correct, that chip making firm will make a fortune. If it's not, they won't.
Re: AI engineers claim new algorithm reduces AI power consumption by 95%
#26The ultimate “you’re doing it wrong”. For he sake of the climate and environment it would be nice to be true. Bad news for Nvidia. “Sell your stock” bad. Does it come with a demonstration?
> Bad news for Nvidia. “Sell your stock” bad.
People say this but then the fastest and most-used implementation of these optimizations is always written in CUDA. If this turns out to not be a hoax, I wouldn't be surprised to see Nvidia prices jump in correlation.
Re: AI engineers claim new algorithm reduces AI power consumption by 95%
#27I posted this about a week ago: https://news.ycombinator.com/item?id=41816598 This has been done for decades in digital circuits, FPGA’s, Digital Signal Processing, etc. Floating point is both resource and power intensive and using FP without the use of dedicated FP processing hardware is something that has been avoided and done without for decades unless absolutely necessary.
a lot of things in the ML research space are rebranding an old concept w a new name as “novel”
Re: AI engineers claim new algorithm reduces AI power consumption by 95%
#28The ultimate “you’re doing it wrong”. For he sake of the climate and environment it would be nice to be true. Bad news for Nvidia. “Sell your stock” bad. Does it come with a demonstration?
Hypothetically, if this is true and simple as the headline implies -- AI using 95% less power doesn't mean AI will use 95% less power, it means we will do 20x more AI. As long as it's the current fad, we will throw as much power and resources at this as we can physically produce, because our economy depends on constant, accelerating growth.
Re: AI engineers claim new algorithm reduces AI power consumption by 95%
#29Isn't this just taking advantage of "log(x) + log(y) = log(xy)"? The IEEE754 floating-point representation stores floats as sign, mantissa, and exponent -- ignore the first two (you quantitized anyway, right?), and the exponent is just an integer storing log() of the float.
Not quite: It's taking advantage of (1+a)(1+b) = 1 + a + b + ab. And where a and b are both small-ish, ab is really small and can just be ignored. So it turns the (1+a)(1+b) into 1+a+b. Which is definitely not the same! But it turns out, machine guessing apparently doesn't care much about the difference.
Plus the 2^-l(m) correction term.
Feels like multiplication shouldn't be needed for convergence, just monotonicity? I wonder how well it would perform if the model was actually trained the same way.
Re: AI engineers claim new algorithm reduces AI power consumption by 95%
#30Extraordinary claims require extraordinary evidence. Maybe it's possible, but consider that some really smart people, in many different groups, have been working diligently in this space for quite a while; so claims of 95% savings on energy costs _with equivalent performance_ is in the extraordinary category. Of course, we'll see when the tide goes out.
The energy claims up to ~70% can be verified. The inference implementation is here: https://github.com/microsoft/BitNet
I'm not an AI person, in any technical sense. The savings being claimed, and I assume verified, are on ARM and x86 chips. The piece doesn't mention swapping mult to add, and a 1-bit LLM is, well, a 1-bit LLM.
Also,
> Additionally, it reduces energy consumption by 55.4% to 70.0%
With humility, I don't know what that means. It seems like some dubious math with percentages.