Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI engineers claim new algorithm reduces AI power consumption by 95%

tomshardware.com

Post reply on HN
Print view View original

81–90 of 174 posts

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#81

Post by etcd »

mouse_ wrote: post #28
andrewstuart wrote: post #15

The ultimate “you’re doing it wrong”. For he sake of the climate and environment it would be nice to be true. Bad news for Nvidia. “Sell your stock” bad. Does it come with a demonstration?

Hypothetically, if this is true and simple as the headline implies -- AI using 95% less power doesn't mean AI will use 95% less power, it means we will do 20x more AI. As long as it's the current fad, we will throw as much power and resources at this as we can physically produce, because our economy depends on constant, accelerating growth.

True. A laptop power pack wattage is probably pretty much unchanged over 30 years for example.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#82

Post by dsv3099i »

mota7 wrote: post #13
remexre wrote: post #2

Isn't this just taking advantage of "log(x) + log(y) = log(xy)"? The IEEE754 floating-point representation stores floats as sign, mantissa, and exponent -- ignore the first two (you quantitized anyway, right?), and the exponent is just an integer storing log() of the float.

Not quite: It's taking advantage of (1+a)(1+b) = 1 + a + b + ab. And where a and b are both small-ish, ab is really small and can just be ignored. So it turns the (1+a)(1+b) into 1+a+b. Which is definitely not the same! But it turns out, machine guessing apparently doesn't care much about the difference.

This trick is used a ton when doing hand calculation in engineering as well. It can save a lot of work.

You're going to have tolerance on the result anyway, so what's a little more error. :)

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#83

Post by l11r »

andrewstuart wrote: post #71
jdiez17 wrote: post #69

Earlier quoted context omitted.

Damn. Seems almost too good to be true. Let’s see where this goes in two weeks.

Intel and AMD will be extremely happy. Nvidia will be very unhappy.

Their GPU will still be needed to do training. As far as I understand this will improve only interference performance and efficiency.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#84

Post by fuzzfactor »

Scene_Cast2 wrote: post #78
fuzzfactor wrote: post #34

Earlier quoted context omitted.

I had a feeling it had to be something like massive waste due to a misguided feature of the algorithms that shouldn't have been there in the first place. Once the "math is done" quite likely it would have paid off better than most investments for the top people to have spent a few short years working with grossly underpowered hardware until they could come up with amazing results there before scaling up. Rather than…

This is a bit like recommending to skip vacuum tubes, think hard and invent transistors.

This is kind of thought-provoking.

That is a good correlation when you think about how much more energy-efficient transistors are than vacuum tubes.

Vacuum tube computers were a thing for a while, but it was more out of desperation than systematic intellectual progress.

OTOH you could look at the present accomplishments like it was throwing more vacuum tubes at a problem that can not be adequately addressed that way.

What turned out to be a solid-state solution was a completely different approach from the ground up.

To the extent a more power-saving technique using the same hardware is only a matter of different software approaches, that would be something that realistically could have been accomplished before so much energy was expended.

Even though I've always thought application-specific circuits would be what really helps ML and AI a lot, and that would end up not being the exact same hardware at all.

If power is truly being wasted enough to start rearing its ugly head, somebody should be able to figure out how to fix it before it gets out-of-hand.

Ironically enough with my experience using vacuum tubes, I've felt that there were some serious losses in technology when the research momentum involved was so rapidly abandoned in favor of "solid-state everything" at any cost.

Maybe it is a good idea to abandon the energy-intensive approaches, as soon as anything completely different that's the least bit promising can barely be seen by a gifted visionary to have a glimmer of potential.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#85

Post by michaelmrose »

fuzzfactor wrote: post #34
crazygringo wrote: post #20

Earlier quoted context omitted.

No. But it does potentially mean that either current or future-tweaked GPUs could run a lot more efficiently -- meaning much faster or with much less energy consumption. You still need the GPU parallelism though.

I had a feeling it had to be something like massive waste due to a misguided feature of the algorithms that shouldn't have been there in the first place. Once the "math is done" quite likely it would have paid off better than most investments for the top people to have spent a few short years working with grossly underpowered hardware until they could come up with amazing results there before scaling up. Rather than…

This comment lives in a fictional world where there is a singular group that could have collectively acted counterfactually. In the real world any actor that individually went this route would have gone bankrupt while the others collected money by showing actual results even if ineffeciently earned.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#86

Post by Randor »

littlestymaar wrote: post #57
Randor wrote: post #19

Earlier quoted context omitted.

The energy claims up to ~70% can be verified. The inference implementation is here: https://github.com/microsoft/BitNet

How does the liked article relate to BitNet at all? It's about the “addition is all you need” paper which AFAIK is unrelated.

Yeah, I get what you're saying but both are challenging the current MatMul methods. The L-Mul paper claims "a power savings of 95%" and that is the thread topic. Bitnet proves that at least 70% is possible by getting rid of MatMul.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#87

Post by giraffe_lady »

jimmaswell wrote: post #72
rossjudson wrote: post #70

Earlier quoted context omitted.

You're sort of presuming that LLMs are going to be a massive service to the world there, aren't you? I think the jury is still out on that one.

They already have been. Even just in programming, even just Copilot has been a life changing productivity booster.

"Even just in programming" the jury is still out. None of my coworkers using these are noticeably more productive than the ones who don't. Outside of programming no one gives a shit except scammers and hype chasers.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#88

Post by Nasrudith »

andrewstuart wrote: post #15

The ultimate “you’re doing it wrong”. For he sake of the climate and environment it would be nice to be true. Bad news for Nvidia. “Sell your stock” bad. Does it come with a demonstration?

Wouldn't reduced power consumption for an unfulfilled demand mean more demand for Nvida as they now need more chips to max out amount of power usage to capacity? (As concentration tends to be the more efficient way.)
Post reply on HN