Live data from Hacker News

New LLM optimization technique slashes memory costs

venturebeat.com

31–40 of 227 posts

Re: New LLM optimization technique slashes memory costs

#31
post #28
post #13

Earlier quoted context omitted.

True. Microsoft's all in, Apple's all in, Nvidia is selling shovels, insurance companies are all in, police & military are all in, education is all in, office management is all in. Who is left to pump line up?

no one is successfully using LLMs for anything other than customer service related things and text generation(coding, writing)

mere trillian dollar industries. so far.

Re: New LLM optimization technique slashes memory costs

#36
Is it possible that after 3-4 years of performance optimizations, both algorithmic and in hardware efficiency, it will turn out that we didn’t really need all of the nuclear plants we’re currently in the process of setting up to satisfy the power demands of AI data centers?

Re: New LLM optimization technique slashes memory costs

#39
post #29

Earlier quoted context omitted.

Google Trends make it seem like we're out of the exponential growth phase for LLMs-- search interest is possibly plateauing. A decline in search interest outside of academia makes sense. The groups who can get by on APIs don't care so much how the sausage is made and just want to see prices come down. Interested parties have likely already found tools that work for them. There's definitely some academic interest outs…

Or maybe it’s Google Search usage that’s plateauing, as LLM interest is answered elsewhere? I am only half kidding.

Well, Google Search trends are also only an imperfect proxy for what we are actually interested in.

Eg tap water is really, really useful and widely deployed. Approximately every household is a user, and that's unlikely to change. But I doubt you'll find much evidence of that in Google Search trends.

Re: New LLM optimization technique slashes memory costs

#40

Is it possible that after 3-4 years of performance optimizations, both algorithmic and in hardware efficiency, it will turn out that we didn’t really need all of the nuclear plants we’re currently in the process of setting up to satisfy the power demands of AI data centers?

It seems (feels?) likely that demand for LLM is elastic, especially when it comes to specialized niche. Less power requirements just mean we run more of them in parallel for stuffs, so the power needs is gonna be growing anyway.
Post reply on HN