Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New LLM optimization technique slashes memory costs

venturebeat.com

Post reply on HN
Print view View original

201–210 of 227 posts

Re: New LLM optimization technique slashes memory costs

#201

Post by cma »

c-linkage wrote: post #145
throwup238 wrote: post #109

Earlier quoted context omitted.

> Constellation closed the adjacent but unconnected Unit 1 reactor in 2019 for economic reasons, but will bring it back to life after signing a 20-year power purchase agreement to supply Microsoft’s energy-hungry data centers, the company announced on Friday. The reactor they’re restarting was operational just five years ago. It’s not a fully decommissioned or melted down reactor and it’s likely all their licensing i…

The largest problem will be finding qualified and vetted personnel. All the people who worked at the plant when it closed five years ago had to find jobs elsewhere. Even though the plant was an important employer in Middletown, I don't know if those former employees will be willing the quit there current jobs to go back, especially if there is a risk the plant will just be shut down again when it once again becomes t…

> especially if there is a risk the plant will just be shut down again when it once again becomes too expensive to operate.

20-year purchase agreement covers half of an adult lifelong career.

Re: New LLM optimization technique slashes memory costs

#202

Post by Eisenstein »

ViewTrick1002 wrote: post #192
Eisenstein wrote: post #191

Earlier quoted context omitted.

'Plenty of places' is not all places and you want to completely count out a significant energy generating ability because you are annoyed that it doesn't agree with your politics. If it isn't feasible then they won't build it -- by going around and advocating against it you are doing the same thing that happened in the 70s and 80s -- removing a perfectly valid option for energy that we need and will otherwise be fulf…

What places aren’t covered by the spectrum with Denmark for higher latitudes and Australia for the near the equator? I’m advocating against wasting public money on nuclear power pretending it is a solution to climate change. Have at it with your own money. I already provided you with the scientists and engineers, but you seem to have completely disregarded them because they did not align with what you wanted. I can d…

I agreed that it costs more and read the study you linked. You are having a hard time accepting that some people might have a different opinion than you and are taking it like they are being obstinate. Sorry it costs more, but I don't think we need to be uniformly opposed to a viable option due to cost.

Re: New LLM optimization technique slashes memory costs

#203

Post by eru »

oblio wrote: post #133
eru wrote: post #49

Earlier quoted context omitted.

Could you please explain? Finance is a big industry, and they are doing lots of different things.

Nobody wants to lose money (savings) or go bankrupt because of hallucinations.

That's one small part of finance. (And essentially solved mechanically with index funds.)

There's a lot more to finance outside of that.

Re: New LLM optimization technique slashes memory costs

#204

Post by eru »

K0balt wrote: post #196
eru wrote: post #49

Earlier quoted context omitted.

Could you please explain? Finance is a big industry, and they are doing lots of different things.

Sure, there’s lots of room for LLMs in helping to do clerical work, HR, that kind of thing. I was actually thinking of the direct management of funds and investments. So yeah, like probably all businesses, the ancillary functions can probably improve productivity using Generative AI with a minimal hit to quality.

You are right about the clerical work, but even pure finance is a lot more than 'direct management of funds and investments'. Have a look at Matt Levine's Money Stuff newsletter for a taste.

And I'm not quite sure why you mention determinism in the grandfather comment? Finance people have been using Monte Carlo simulations for ages. (And removing non-determinism from LLMs by fixing the seed of any pseudo-random number generator used wouldn't really change anything, would it?)

Re: New LLM optimization technique slashes memory costs

#205

Post by h0l0cube »

Auracle wrote: post #52
h0l0cube wrote: post #41

Earlier quoted context omitted.

Oh, I don’t know, how about reducing the search space/accelerating the search speed for potential room temperature superconductors? Or how about the same for viable battery chemistries?

It’s a good thing humanity can multitask.

Yeah, but it’s not an even sharing of resources. LLMs are consuming a vast amount of human attention (no pun intended) at the expense of technological pursuits that will more certainly generate value. As far as can be told LLMs are reaching a plateau in terms of real world value, batteries and superconductors have calculably more potential.

Re: New LLM optimization technique slashes memory costs

#206

Post by h0l0cube »

bubaumba wrote: post #69
h0l0cube wrote: post #41

Earlier quoted context omitted.

Oh, I don’t know, how about reducing the search space/accelerating the search speed for potential room temperature superconductors? Or how about the same for viable battery chemistries?

> search speed for potential room temperature superconductors? and what if it's a dead end?

What if LLMs are a dead end?

Re: New LLM optimization technique slashes memory costs

#207

Post by portaouflop »

Eisenstein wrote: post #202
ViewTrick1002 wrote: post #192

Earlier quoted context omitted.

What places aren’t covered by the spectrum with Denmark for higher latitudes and Australia for the near the equator? I’m advocating against wasting public money on nuclear power pretending it is a solution to climate change. Have at it with your own money. I already provided you with the scientists and engineers, but you seem to have completely disregarded them because they did not align with what you wanted. I can d…

I agreed that it costs more and read the study you linked. You are having a hard time accepting that some people might have a different opinion than you and are taking it like they are being obstinate. Sorry it costs more, but I don't think we need to be uniformly opposed to a viable option due to cost.

I genuinely don’t understand why you think nuclear is a viable alternative.

You agree it costs more, is less flexible, takes longer to be operational than renewable energies.

You didn’t present any good argument for nuclear except “something something you don’t like my politics”

I agree with GP that nuclear is often used as a smokescreen to delay doing __anything__ practical and instead keep burning coal etc

Re: New LLM optimization technique slashes memory costs

#208

Post by h0l0cube »

pizza wrote: post #79
h0l0cube wrote: post #41

Earlier quoted context omitted.

Oh, I don’t know, how about reducing the search space/accelerating the search speed for potential room temperature superconductors? Or how about the same for viable battery chemistries?

Would you like that with or without tokens?

Go ask ChatGPT how to make a room temperature superconductor

Re: New LLM optimization technique slashes memory costs

#209

Post by mewpmewp2 »

bongodongobob wrote: post #197
mewpmewp2 wrote: post #129

Earlier quoted context omitted.

But obviously it would be far from accuracy that LLM would be able to do. E.g. generate search keywords, tags, other type of meta data for a certain document.

Yup that's exactly it. By being able to tag things with all sorts of in house meta data we were then able to search and group things extremely accurately. There was still a lot of human in the mix, but this made the whole task going from "idk if we can even consider doing this" to "great, we can break this down and chip away at it over the next few months/throw some interns at it".

Yeah, I don't know - hearing arguments that this was already done by ML algorithms is to me hearing like "moving from place A to B existed already before cars". But it seems like a common sentiment. So much that simple ML attempted to be doing required massive amount of training and training data specific to your domain before you could use it, and LLM can do it out of the box, and actually consider nuance.

I think organizing and structuring data from unorganized data from the past is a massive use case that seems heavily underrated by so many right now. People spend a lot of time on figuring out where to find some data, internally in companies, etc.

Re: New LLM optimization technique slashes memory costs

#210

Post by exe34 »

portaouflop wrote: post #207
Eisenstein wrote: post #202

Earlier quoted context omitted.

I agreed that it costs more and read the study you linked. You are having a hard time accepting that some people might have a different opinion than you and are taking it like they are being obstinate. Sorry it costs more, but I don't think we need to be uniformly opposed to a viable option due to cost.

I genuinely don’t understand why you think nuclear is a viable alternative. You agree it costs more, is less flexible, takes longer to be operational than renewable energies. You didn’t present any good argument for nuclear except “something something you don’t like my politics” I agree with GP that nuclear is often used as a smokescreen to delay doing __anything__ practical and instead keep burning coal etc

in practice, 20 years of walking away from nuclear meant that Germany brought coal-fired stations back last year. I'm sure renewables will stop it happening again in 20 years _this time_.
Post reply on HN