Live data from Hacker News

AI engineers claim new algorithm reduces AI power consumption by 95%

tomshardware.com

151–160 of 174 posts

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#151

Earlier quoted context omitted.

I've been using copilot for several months. If I could figure out a way to measure its impact on my productivity, I'd probably see a single digit percentage boost in "productivity". This is not life-changing for me. And for some tasks, it's actually worse than nothing. As in, I spend time feeding it a task, and it just completely fails to do anything useful.

If you're already a competent developer, I think that's a reasonable expectation of impact on productivity. I think the 'life-changing' part comes in helping someone get to the point of building things with code where before they couldn't (or believed they couldn't). It does a lot better job of turning the enthusiasts and code-curious into amateurs vs. empowering professionals.

> turning the enthusiasts and code-curious into amateurs vs. empowering professionals.

I'm firmly in #2. My other comment goes over how.

I'm intrigued to see how devs in #1 grow. One might be wary those devs would grow into bad habits and not thinking for themselves, but it might be a case of the ancient Greek rant against written books hindering memorization. Could be that they'll actually grow to be even better devs unburdened by time wasted on trivial details.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#152
post #97

Earlier quoted context omitted.

If 20 mins of informations can legitimately be condensed into 20 seconds, it sounds like the original wasn't worth reading in the first place. Could have skipped the llm entirely.

Think of the summary of a zoom call. Or of a chapter that you're not sure if you care to read or not. Not all content is worth consuming, and not all content is dense.

If I had a recording of the zoom call I could generate a summary on demand with better tools than were available at the time the zoom call was made.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#153
post #148

Earlier quoted context omitted.

> Not on HN. Customary is to use > paragraph quotes like you did. However I will keep that in mind. Hacker News is not some strange place where the normal rules of discourse don't apply. I assume you are familiar with the function of quotation marks. > If we're both grading a single student (LLM) in same field (programming), and you find it great and I find it disappointing, it means one of us is scoring it wrong. No…

> Hacker News is not some strange place where the normal rules of discourse don't apply. I assume you are familiar with the function of quotation marks. Language is all about context. I wasn't trying to be deceitful. And on HN I've never seen anyone using quotation marks to quote people. > Writing good documentation is certainly a challenging task. Doctests isn't same as writing documentation. Doctest are the simples…

> Language is all about context. I wasn't trying to be deceitful. And on HN I've never seen anyone using quotation marks to quote people.

It's still unclear how this apparent lack of knowledge of basic writing mechanics would justify your use of quotation marks to attempt a straw man argument wherein you deliberately attempted to convince me that OP said something completely different.

> Doctests isn't same as writing documentation. Doctest are the simplest form of documentation. Given function named so and so write API doc + example. It could not even write example that passed syntax check.

That truly sounds like a skill issue. This no-true-Scotsman angle is silly. I said documentation and tests, I don't know how you got "doctests" out of that. I said "documentation", and "tests". I didn't say "the simplest form of documentation", that is another straw man on your behalf.

> Then you haven't given it interesting/complex enough problems.

Wow, the arrogance. There is absolutely nothing to justify this assumption. It's exceedingly likely that you yourself aren't capable of interacting meaningfully with LLMs for one reason or another, not that I haven't considered interesting or complex problems. I bring some extraordinarily difficult cross-domain problems to these tools and end up satisfied with the results far more often than not.

My argument is literally that cutting-edge LLMs excel with complex problems, and they do in many cases in the right hands. It's unfortunate if you can't find these problems "interesting" enough, but that hasn't stopped me from getting good enough results to justify using an LLM during research and development.

> Also this isn't about practice. It's about its capabilities.

Unfortunately, this discourse has made it clear that you do need considerable practice, because you seem to get bad results, and you're more interested in defending those bad results even if it means insulting others, instead of just considering that you might not quite be skilled enough.

> This is the stochastic parrot in action. Why? Because it tried to pass of JSON like structure as YAML.

That proves its stochasticity, but it doesn't prove it is a "stochastic parrot". As long as you lack the capability to realistically assess these models, it's no wonder that you've had such bad experiences. You didn't even bother clarifying which LLM you used, nor did you mention any parameters of your experiment or even if you attempted multiple trials with different LLMs or prompts. You failed to follow the scientific method and so it's no surprise that you got subpar results.

> Whatever LLM's are they aren't intelligent.

You have demonstrated throughout this discussion that you aren't capable of assessing machine intelligence. If you learned how to be more open-minded and took the time to learn more about these new technologies, instead of complaining about contemporary shortcomings and bashing those who do benefit from the technologies, it would likely open many doors for you.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#154

Earlier quoted context omitted.

> I almost forgot how much it sucks having to jump to google every other minute Even allowing for some hyperbole, your programming experience is extremely different from mine. Looking anything up outside the IDE, let alone via Google, is by far the exception for me rather than the rule. I've long suspected that this kind of difference explains a lot of the difference in how Copilot is perceived.

Claiming LLMs are a massive boost for coding productivity is becoming a red flag that the claimant has a tenuous grasp on the skills necessary. Yeah if you have to look up everything all the time and you can't tell the AI slop isn't very good, you can put out code quite fast.

Hey, we were all beginners once!

On another note, even if you are experienced it helps when doing new stuff and you don’t know the proper syntax for what you want. For example let’s say your using flutter, you can just type

// bold

And it will help put the proper bold stuff in there.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#155

Earlier quoted context omitted.

You're sort of presuming that LLMs are going to be a massive service to the world there, aren't you? I think the jury is still out on that one.

They already have been. Even just in programming, even just Copilot has been a life changing productivity booster.

actually, studies seem to show it makes code worse. Just like llms can confidently spout junk, devs using llms confidently check in more bugs.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#156
post #7

https://arxiv.org/abs/2410.00907 ABSTRACT Large neural networks spend most computation on floating point tensor multiplications. In this work, we find that a floating point multiplier can be approximated by one integer adder with high precision. We propose the linear-complexity multiplication (L-Mul) algorithm that approximates floating point number multiplication with integer addition operations. The new algorithm c…

Is this effectively quantizing without actually quantizing?

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#157
post #7

https://arxiv.org/abs/2410.00907 ABSTRACT Large neural networks spend most computation on floating point tensor multiplications. In this work, we find that a floating point multiplier can be approximated by one integer adder with high precision. We propose the linear-complexity multiplication (L-Mul) algorithm that approximates floating point number multiplication with integer addition operations. The new algorithm c…

Does this mean you can train efficiently without GPUs? Presumably there will be a lot of interest.

The GPU main advantage is its parallelism - thousands of cores compared handful of cores in CPUs.

If you're training models with billions of parameters, you're still gonna need that.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#158

Earlier quoted context omitted.

I've been using it for over a year I think. I don't often feed it tasks with comments so much as go about things the same as usual and let it autocomplete. The time and cognitive load saved adds up massively. I've had to go without it for a bit while my workplace gets its license in order for the corporate version and the personal version has an issue with the proxy, and it's been agonizing going without it again. I…

I don't know about you but LLMs spit out garbage nonsense frequent enough that I can't trust their output in any context I cannot personally verify the validity of.

Same here. The Cody autocomplete is so off base all the time that it’s deactivated.

I serve Cody a direct question about 1-3 times a week. Of that, it gets maybe 50% correct on the first try. I don’t bother with a second try because by then I’ve already spent the equivalent amount of time looking at the relevant library source code and/or docs would have taken.

Re: AI engineers claim new algorithm reduces AI power consumption by 95%

#160
post #148

Earlier quoted context omitted.

> Hacker News is not some strange place where the normal rules of discourse don't apply. I assume you are familiar with the function of quotation marks. Language is all about context. I wasn't trying to be deceitful. And on HN I've never seen anyone using quotation marks to quote people. > Writing good documentation is certainly a challenging task. Doctests isn't same as writing documentation. Doctest are the simples…

> Language is all about context. I wasn't trying to be deceitful. And on HN I've never seen anyone using quotation marks to quote people. It's still unclear how this apparent lack of knowledge of basic writing mechanics would justify your use of quotation marks to attempt a straw man argument wherein you deliberately attempted to convince me that OP said something completely different. > Doctests isn't same as writin…

> That truly sounds like a skill issue. This no-true-Scotsman angle is silly. I said documentation and tests, I don't know how you got "doctests" out of that. I said "documentation", and "tests". I didn't say "the simplest form of documentation", that is another straw man on your behalf.

What are you on about? Doctest is the simplest form of documentation and test. I.e. you don't have to write an in-depth test, you just need to understand what the function does. I expect even juniors can write a doctest that passes the compiler check. Not a good, not a passing one doctest, a COMPILING one. It's rate of writing a passing one was even worse.

> Wow, the arrogance. There is absolutely nothing to justify this assumption.

Ok, then. Prove what exactly hard problems did you give it?

I gave my examples, I noticed it fails at complex tasks like YAML parser in an unknown language.

I noticed when confronted with anything harder than writing pure boilerplate, it fails. E.g. it would fail 10% of the time.

> Unfortunately, this discourse has made it clear that you do need considerable practice

You can practice with a stochastic parrot all you want, it won't make it an Einstein. Programming is all about converting requirements to math, and LLMs aren't good at it. Do I need to link stuff like doing basic calculation and counting 'r' in the word 'strawberries'.

The best you can do is half the error rate, but that follows a power law. You need to double the energy to half the error rate. So unless you intend to boil the surface of the Earth to get it to be decent at programming, I don't think it's going to change anytime soon.

> You have demonstrated throughout this discussion that you aren't capable of assessing machine intelligence.

Pure ad hominem. You've demonstrated nothing outside your ""Trust me bro, it's not a bubble"" and ""You're wrong"". I'm using double double quotes so you don't assume I'm quoting you.

> You didn't even bother clarifying which LLM you used.

For YAML parser, I used Chat GPT-4o at my friend's place. For the rest of the tasks I used JetBrains AI assistant, which is a mix of Chat GPT-4, GPT-4o and GPT-3.

Post reply on HN