Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI engineers claim new algorithm reduces AI power consumption by 95%

tomshardware.com

Post reply on HN
Print view View original

91–100 of 174 posts

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#91

Post by recursive »

jimmaswell wrote: post #72
rossjudson wrote: post #70

Earlier quoted context omitted.

You're sort of presuming that LLMs are going to be a massive service to the world there, aren't you? I think the jury is still out on that one.

They already have been. Even just in programming, even just Copilot has been a life changing productivity booster.

I've been using copilot for several months. If I could figure out a way to measure its impact on my productivity, I'd probably see a single digit percentage boost in "productivity". This is not life-changing for me. And for some tasks, it's actually worse than nothing. As in, I spend time feeding it a task, and it just completely fails to do anything useful.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#92

Post by AYBABTME »

wruza wrote: post #76
jimmaswell wrote: post #72

Earlier quoted context omitted.

They already have been. Even just in programming, even just Copilot has been a life changing productivity booster.

Are you sure it’s a life changing productivity booster? Sometimes I look at my projects and wonder how would I explain it to an LLM what this code should have done if it didn’t exist yet. Must be a shitton of boilerplate programming for copilot to be a life-changing experience.

You haven't used them enough. Everytime an LLM reduces my search from 1min to 5s, the LLM pays.

Just summary features: save me 20min of reading a transcript, turn it into 20s. That's a huge enabler.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#93

Post by ksec »

manquer wrote: post #22
kayo_20211030 wrote: post #10

Extraordinary claims require extraordinary evidence. Maybe it's possible, but consider that some really smart people, in many different groups, have been working diligently in this space for quite a while; so claims of 95% savings on energy costs _with equivalent performance_ is in the extraordinary category. Of course, we'll see when the tide goes out.

It is a click bait headline the claim itself is not extraordinary. the preprint from arxiv was posted here some time back . The 95% gains is specifically only for multiplication operations, inference is compute light and memory heavy in the first place so the actual gains would be far less smaller . Tech journalism (all journalism really) can hardly be trusted to publish grounded news with the focus on clicks and rev…

>Tech journalism (all journalism really) can hardly be trusted to publish grounded news with the focus on clicks and revenue they need to survive.

Right now the only way to gain real knowledge is actually to read comments of those articles.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#95

Post by newyankee »

michaelmrose wrote: post #85
fuzzfactor wrote: post #34

Earlier quoted context omitted.

I had a feeling it had to be something like massive waste due to a misguided feature of the algorithms that shouldn't have been there in the first place. Once the "math is done" quite likely it would have paid off better than most investments for the top people to have spent a few short years working with grossly underpowered hardware until they could come up with amazing results there before scaling up. Rather than…

This comment lives in a fictional world where there is a singular group that could have collectively acted counterfactually. In the real world any actor that individually went this route would have gone bankrupt while the others collected money by showing actual results even if ineffeciently earned.

Also it is likely that the rise of LLMs gave many researchers in allied fields the impetus to tackle with the problems that are relevant to making it more efficient and people stumbled upon a solution hiding there.

The momentum with LLMs and allied technology may last till it keeps on improving even by a few percentage points and keeps shattering human created new benchmarks every few months

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#97

Post by mecsred »

AYBABTME wrote: post #92
wruza wrote: post #76

Earlier quoted context omitted.

Are you sure it’s a life changing productivity booster? Sometimes I look at my projects and wonder how would I explain it to an LLM what this code should have done if it didn’t exist yet. Must be a shitton of boilerplate programming for copilot to be a life-changing experience.

You haven't used them enough. Everytime an LLM reduces my search from 1min to 5s, the LLM pays. Just summary features: save me 20min of reading a transcript, turn it into 20s. That's a huge enabler.

If 20 mins of informations can legitimately be condensed into 20 seconds, it sounds like the original wasn't worth reading in the first place. Could have skipped the llm entirely.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#98

Post by jimmaswell »

recursive wrote: post #91
jimmaswell wrote: post #72

Earlier quoted context omitted.

They already have been. Even just in programming, even just Copilot has been a life changing productivity booster.

I've been using copilot for several months. If I could figure out a way to measure its impact on my productivity, I'd probably see a single digit percentage boost in "productivity". This is not life-changing for me. And for some tasks, it's actually worse than nothing. As in, I spend time feeding it a task, and it just completely fails to do anything useful.

I've been using it for over a year I think. I don't often feed it tasks with comments so much as go about things the same as usual and let it autocomplete. The time and cognitive load saved adds up massively. I've had to go without it for a bit while my workplace gets its license in order for the corporate version and the personal version has an issue with the proxy, and it's been agonizing going without it again. I almost forgot how much it sucks having to jump to google every other minute, and it was easy to start to take for granted how much context copilot was letting me not have to hold onto in my head. It really lets me work on the problem as opposed to being mired in immaterial details. It feels like I'm at least 2x slower overall without it.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#99

Post by Art9681 »

In the end the power consumption means the current models that are "good enough" will fit a much smaller compute budget such as edge devices. However, enthusiasts are still going to want the best hardware they can afford because inevitably, everyone will want to maximize the size and intelligence of a model they can run. So we're just going to scale. This might bring a GPT-4 level to edge devices, but we are still going to want to run what might resemble a GPT-5/6 model on the best hardware possible at the time. So don't throw away your GPU's yet. This will bring capabilities to mass market, but your high end GPU will still scale the solution n-fold and youll be able to run models with disregard to the energy savings promoted in the headline.

In other sensationalized words: "AI engineers can claim new algorithm allows them to fit GPT-5 in an RTX5090 running at 600 watts."

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#100

Post by quantadev »

chx wrote: post #54
quantadev wrote: post #44

I wonder if someone has feed this entire "problem" into the latest Chat GPT-01 (the new model with reasoning capability), and just fed it in all the code for a Multilayer Perceptron and then given it the task/prompt of finding ways to implement the same network using only integer operations. Surely even the OpenAI devs must have done this like the minute they got done training that model, right? I wonder if they'd ev…

You are imaging LLMs are capable of much more than they actually are. Here's the only thing they are good for. https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natur…

No, I'm not imagining things. You are, however, imaging (incorrectly) that I'm not an expert with AI who's already seen superhuman performance out of LLM prompts in the vast majority of every software development question I've ever asked them, starting all the way back at GPT-3.5.
Post reply on HN