Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New LLM optimization technique slashes memory costs

venturebeat.com

Post reply on HN
Print view View original

221–227 of 227 posts

Re: New LLM optimization technique slashes memory costs

#221

Post by pizza »

h0l0cube wrote: post #220
pizza wrote: post #213

Earlier quoted context omitted.

Ok, done. I can report to you that it helped me cut down my personal search space. Imagine what such a tool could do in the hands of a subject matter expert with rudimentary critical thinking ability and the faintest hint of a grasp of using the scientific method to verify claims, wow..

> Imagine what such a tool could do in the hands of a subject matter expert Nothing. Because LLMs can’t spit out anything more than the corpus of information they’ve consumed. LLMs aren’t AGI.

You're making a fundamental error in your reasoning. An LLM's training corpus being fixed doesn't limit the system's total information processing capability when used as a tool by a human researcher. While the LLM itself can't generate truly novel information (per the data processing inequality), a human researcher using it as a dynamic search and analysis tool can absolutely generate new insights and discoveries through their interaction with it. The human-LLM system is open, not closed.

This is analogous to how a calculator cannot output any number that isn't computationally derivable from its programming, yet humans using calculators have discovered new mathematical proofs. The tool augments human capability without being AGI.

Your argument is essentially claiming that because a microscope can't generate new cellular structures, it can't help biologists make new discoveries.

Re: New LLM optimization technique slashes memory costs

#222

Post by ViewTrick1002 »

exe34 wrote: post #219
portaouflop wrote: post #216

Earlier quoted context omitted.

What answer do you expect here? This is not something you can answer clearly, no one can. I personally would say since renewables (there are many different types of renewable energy sources btw) are so much cheaper and easier to build they are more consistent. France for example has really shitty nuclear plants that have been falling apart since the 90s - they are not reliable and fixing them is not feasible

the correct answer is no. solar doesn't work very well at night. wind isn't always blowing.

You seem to not be very updated, all the while holding extremely strong beliefs.

Storage delivering nuclear scale energy day in and day out in California:

https://blog.gridstatus.io/caiso-batteries-apr-2024/

Storage plummeting in cost 20% YoY, now at $66/kWh.

https://reneweconomy.com.au/mind-blowing-battery-cell-prices...

Re: New LLM optimization technique slashes memory costs

#223

Post by eru »

oblio wrote: post #215
eru wrote: post #203

Earlier quoted context omitted.

That's one small part of finance. (And essentially solved mechanically with index funds.) There's a lot more to finance outside of that.

At the end of the day, the hard limit in finance is defaulting. Everything outside that is financial poetry (or engineering :-p). I know every segment of finance loves to pretend that's not the case, because their jobs (and high salaries) frequently rely on that not being true (see the subprime mortgage crisis).

> At the end of the day, the hard limit in finance is defaulting. Everything outside that is financial poetry (or engineering :-p).

You are forgetting all about regulations and taxation (and how to work with / around them). And how to cleverly read documents, and exploit loop holes in contracts.

There's so much more to finance.

(And for eg stocks or commodities, there's not even any notion of defaulting. Defaulting only really makes sense when you have fixed obligations. 'Fixed income' is only one part of finance.)

> (see the subprime mortgage crisis)

That's actually a more nuanced topic than you think. See eg https://kevinerdmann.substack.com/p/subprime-bank-runs-and-t... and other posts by Kevin Erdmann on the topic.

Re: New LLM optimization technique slashes memory costs

#224

Post by h0l0cube »

pizza wrote: post #221
h0l0cube wrote: post #220

Earlier quoted context omitted.

> Imagine what such a tool could do in the hands of a subject matter expert Nothing. Because LLMs can’t spit out anything more than the corpus of information they’ve consumed. LLMs aren’t AGI.

You're making a fundamental error in your reasoning. An LLM's training corpus being fixed doesn't limit the system's total information processing capability when used as a tool by a human researcher. While the LLM itself can't generate truly novel information (per the data processing inequality), a human researcher using it as a dynamic search and analysis tool can absolutely generate new insights and discoveries thr…

Did an AI write this?

You have a fundamental misunderstanding of how LLMs work, which is why you think they are magical.

Of course if you play the LLM Pachinko machine you can get all sorts of novel output from it, but it’s only useful for certain tasks. It’s great for translation, summarizing (also a kind of translation), and to some degree it can recall from its training corpus an interesting fact. And yes, it can synthesize novel content such as poetry, or adapt an oft-used coding pattern in a flavor specified by a prompt.

What it can’t do is come up with a new idea. At least not in a way better than rolling a dice. It may come up with an idea that you, dear reader, may not have encountered, which makes it great for education.

I don’t have anything more to say, but you’re welcome to continue this discussion with an agent of your choice.

Re: New LLM optimization technique slashes memory costs

#225

Post by pizza »

h0l0cube wrote: post #224
pizza wrote: post #221

Earlier quoted context omitted.

You're making a fundamental error in your reasoning. An LLM's training corpus being fixed doesn't limit the system's total information processing capability when used as a tool by a human researcher. While the LLM itself can't generate truly novel information (per the data processing inequality), a human researcher using it as a dynamic search and analysis tool can absolutely generate new insights and discoveries thr…

Did an AI write this? You have a fundamental misunderstanding of how LLMs work, which is why you think they are magical. Of course if you play the LLM Pachinko machine you can get all sorts of novel output from it, but it’s only useful for certain tasks. It’s great for translation, summarizing (also a kind of translation), and to some degree it can recall from its training corpus an interesting fact. And yes, it can…

just read the literature arxiv.org/abs/2410.01720

Re: New LLM optimization technique slashes memory costs

#226

Post by cratermoon »

sobani wrote: post #217
adelpozo wrote: post #173

Earlier quoted context omitted.

Yeah, but what I found thought provoking is what if you send the solar panels and the datacenter as well for training. No need to transmission of power down to earth. I guess then it becomes a heat dissipation and hardware upgrade and maintenance. But again, thought provoking.

Heat dissipation becomes a _huge_ problem when you deploy a data center inside a perfect insulator, the vacuum of space. Currently about a third of the energy consumption of a data center spent on cooling (heat dissipation)? And that's with the use of a huge heat sink, the earth.

Plus, I feel like GP hasn't ever seen an actual data center. One does not simply strap on on top of a rocket (even a SpaceX Starship) and toss it into LEO.

Re: New LLM optimization technique slashes memory costs

#227

Post by bubaumba »

h0l0cube wrote: post #206
bubaumba wrote: post #69

Earlier quoted context omitted.

> search speed for potential room temperature superconductors? and what if it's a dead end?

What if LLMs are a dead end?

Nothing to worry about. They are dead end only if we find something better. Till then they here to stay. And likely even after they will be used.
Post reply on HN