AI engineers claim new algorithm reduces AI power consumption by 95% • Hacker News Board

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#171

Post by yogrish » Wed, Oct 23, 2024, 4:20 AM UTC

djoldman wrote: post #7

https://arxiv.org/abs/2410.00907 ABSTRACT Large neural networks spend most computation on floating point tensor multiplications. In this work, we find that a floating point multiplier can be approximated by one integer adder with high precision. We propose the linear-complexity multiplication (L-Mul) algorithm that approximates floating point number multiplication with integer addition operations. The new algorithm c…

we used to use Fixed point multiplications (Q Format) in DSP algorithms on different DSP architectures. https://en.wikipedia.org/wiki/Q_(number_format). They used to be so fast and near accurate to floating point multiplications. Probably we need to use those DSPs blocks as part of Tensors/GPUs to realise both fast multiplications & parallelisms.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#172

Post by bicepjai » Wed, Oct 23, 2024, 2:32 PM UTC

mattxxx wrote: post #75

GistNoesis wrote: post #36

Does https://en.wikipedia.org/wiki/Jevons_paradox apply in this case ?

That's interesting. Obviously, energy cost creates a barrier to entry, so reduction of cost reduces the barrier to entry... which adds more players... which increases demand.

This is why I love HN

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#173

Post by robomartin » Tue, Oct 29, 2024, 2:35 AM UTC

ujikoluk wrote: post #18

robomartin wrote: post #9

I posted this about a week ago: https://news.ycombinator.com/item?id=41816598 This has been done for decades in digital circuits, FPGA’s, Digital Signal Processing, etc. Floating point is both resource and power intensive and using FP without the use of dedicated FP processing hardware is something that has been avoided and done without for decades unless absolutely necessary.

Explain more for the uninitiated please.

Not sure there's much to explain. Using integers for math in digital circuits is far more resource and computationally efficient than floating-point math. It has been decades since I did the math on the difference. I'll just guess that it could easily be an order of magnitude better across both metrics.

At basic level it is very simple: A 10 bit bus gives you the ability to represent numbers between 0 and 1 with a resolution of approximately 0.001. 12 bits would be four times better. Integer circuits can do the math in one clock cycle. Hardware multipliers do the same. To rescale the numbers after multiplication you just take the N high bits, where N is your bus width; which is a zero clock-cycle operation. Etc.

In training a neural network, the back propagation math can be implemented using almost the same logic used for a polyphase FIR filter.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#174

Post by [deleted] » Fri, Nov 01, 2024, 2:18 PM UTC

Ygg2 wrote: post #160

soulofmischief wrote: post #153

Earlier quoted context omitted.

> Language is all about context. I wasn't trying to be deceitful. And on HN I've never seen anyone using quotation marks to quote people. It's still unclear how this apparent lack of knowledge of basic writing mechanics would justify your use of quotation marks to attempt a straw man argument wherein you deliberately attempted to convince me that OP said something completely different. > Doctests isn't same as writin…

> That truly sounds like a skill issue. This no-true-Scotsman angle is silly. I said documentation and tests, I don't know how you got "doctests" out of that. I said "documentation", and "tests". I didn't say "the simplest form of documentation", that is another straw man on your behalf. What are you on about? Doctest is the simplest form of documentation and test. I.e. you don't have to write an in-depth test, you j…

[deleted]