Live data from Hacker News

New LLM optimization technique slashes memory costs

venturebeat.com

131–140 of 227 posts

Re: New LLM optimization technique slashes memory costs

#131
post #80

Earlier quoted context omitted.

See also https://en.m.wikipedia.org/wiki/Induced_demand

How do you argue that demand was induced as opposed to existing demand served?

Because economists only call it demand to the extent that people are willing and able to make a purchase.

If someone has a need or wants something real bad but can't afford to buy the desired quantity at the prevailing price then economists don't call it demand.

Re: New LLM optimization technique slashes memory costs

#132
post #80

Earlier quoted context omitted.

See also https://en.m.wikipedia.org/wiki/Induced_demand

How do you argue that demand was induced as opposed to existing demand served?

Maybe because of an extremely aggressive marketing, and pushing "AI features" into literally everything, and there being a pushback against that?

Re: New LLM optimization technique slashes memory costs

#133
post #49
post #25

Earlier quoted context omitted.

Finance is basically all of the reasons not to use (generative, LLM based) AI , all in one vertical. The poster child of determinism.

Could you please explain? Finance is a big industry, and they are doing lots of different things.

Nobody wants to lose money (savings) or go bankrupt because of hallucinations.

Re: New LLM optimization technique slashes memory costs

#134
post #82

It’s mind bogglingly crazy that language models rivaling ones that used to require huge GPUs with a ton of VRAM to run now run on my upper-mid-range laptop from 4 years ago. At usable speed. Crazy. I didn’t expect capable language models to be practical/possible to run loyally, much less on hardware I already have.

You have a sota multi-modal LLM running in your head at 20W, shared with best in class sensor package and top performing robotics control unit. There’s soooo much more to optimize.

I would argue that our sensor package is losing its lead very quickly-- audio performance is already on par with current tech and image processing is closing the gap very quickly as well (it helps a lot that silicon-based technology is much less constrained on bandwidth). Tactile sensing is still lightyears ahead, and I don't see that situation improving anytime soon...

Re: New LLM optimization technique slashes memory costs

#135
post #75
post #48

Earlier quoted context omitted.

Nobody is building nuclear power plants for data centres. A few people have signed some paperwork saying that they would buy electricity from new nuclear plants if they could deliver it at a certain price, a price mind you that has not been done before. Others are trying to restart an existing reactor at three mile island (a thing that has never been done before, and likely won't be done now since the reactor was shu…

> Nobody is building nuclear power plants for data centres. A few people have signed some paperwork saying that they would buy electricity from new nuclear plants if they could deliver it at a certain price, a price mind you that has not been done before. Not building new, but I think Microsoft paying to restart a reactor at Three Mile Island for their datacenter is much more significant than you make the deals sound…

Microsoft isn’t paying to restart. A PPA is a contract saying they will purchase electricity for a specified price for a fixed term. Three Mile Island needs to be able to produce the electricity for the specified price for Microsoft to pay buy it. If it’s above that price Microsoft is off the hook.

Re: New LLM optimization technique slashes memory costs

#136
post #48

Earlier quoted context omitted.

Nobody is building nuclear power plants for data centres. A few people have signed some paperwork saying that they would buy electricity from new nuclear plants if they could deliver it at a certain price, a price mind you that has not been done before. Others are trying to restart an existing reactor at three mile island (a thing that has never been done before, and likely won't be done now since the reactor was shu…

Could be a good candidate for factobattery. Overbuild the system, run them at full speed at peak solar generation, then underclock them at night. https://www.moderndescartes.com/essays/factobattery/

This is a really interesting idea! Of course, in practice that will just mean crypto-token mining rather than anything useful.

Re: New LLM optimization technique slashes memory costs

#137
post #76
post #48

Earlier quoted context omitted.

Nobody is building nuclear power plants for data centres. A few people have signed some paperwork saying that they would buy electricity from new nuclear plants if they could deliver it at a certain price, a price mind you that has not been done before. Others are trying to restart an existing reactor at three mile island (a thing that has never been done before, and likely won't be done now since the reactor was shu…

Look, I agree that nuclear is difficult, but Google and Microsoft have publicly committed to those projects you’re mentioning. I don’t understand your dismissive tone that all of it is hogwash? This is one of those HN armchair comments.

My tone is because this is a simple predatory delay strategy.

Tomorrow, tomorrow, I’ll decarbonize tomorrow.

Instead of paying to buy wind and solar plants, which can go up today they are signing a meaningless agreement for the future.

A PPA isn’t worth the paper it’s written on if the seller can’t produce electricity at the agreed upon price by the date required.

Take Three Mile Island. It was closed in 2019 since it was uneconomical to run. Since then renewables have continued getting substantially cheaper, while the reactor has been in the process of decommissioning.

Instead of spending money on building wind and solar, Microsoft saw how well Vogtle went and decided that another first of it’s kind nuclear project is the best way to make it appear like they’re doing something.

Re: New LLM optimization technique slashes memory costs

#138
post #76

Earlier quoted context omitted.

Look, I agree that nuclear is difficult, but Google and Microsoft have publicly committed to those projects you’re mentioning. I don’t understand your dismissive tone that all of it is hogwash? This is one of those HN armchair comments.

My tone is because this is a simple predatory delay strategy. Tomorrow, tomorrow, I’ll decarbonize tomorrow. Instead of paying to buy wind and solar plants, which can go up today they are signing a meaningless agreement for the future . A PPA isn’t worth the paper it’s written on if the seller can’t produce electricity at the agreed upon price by the date required. Take Three Mile Island. It was closed in 2019 since…

I've been told my entire life that it's too late for nuclear, we should have been building them 20 years ago.

I think now's fine, even if it takes time. these companies already buy a ton of power from renewable sources, and it's good to diversify - nuclear is a good backup to have.

Re: New LLM optimization technique slashes memory costs

#140
post #138

Earlier quoted context omitted.

My tone is because this is a simple predatory delay strategy. Tomorrow, tomorrow, I’ll decarbonize tomorrow. Instead of paying to buy wind and solar plants, which can go up today they are signing a meaningless agreement for the future . A PPA isn’t worth the paper it’s written on if the seller can’t produce electricity at the agreed upon price by the date required. Take Three Mile Island. It was closed in 2019 since…

I've been told my entire life that it's too late for nuclear, we should have been building them 20 years ago. I think now's fine, even if it takes time. these companies already buy a ton of power from renewable sources, and it's good to diversify - nuclear is a good backup to have.

The west tried building nuclear power 20 years ago. If it had delivered we would be building more now.

It did not deliver. It is time to leave nuclear power to the past just like we have done with the steam engine.

It had its heyday but better cheaper technology replaced it.

Post reply on HN