Live data from Hacker News

New LLM optimization technique slashes memory costs

venturebeat.com

141–150 of 227 posts

Re: New LLM optimization technique slashes memory costs

#141

Earlier quoted context omitted.

There's no paradox in that. People became more capable and can afford to do more.

Also I think this will play out for AI as a productivity multiplier. Instead of people having less work there will be more to do since more things are worth doing now. For the following few years at least.

Work expands to fill the available time, after all.

Re: New LLM optimization technique slashes memory costs

#142

Earlier quoted context omitted.

How do you argue that demand was induced as opposed to existing demand served?

Yes, the term is a bit clumsy. The way I think of it, people have desires (to drive on the highway), but are dissuaded from doing so by disincentives (it’s too busy). Adding a lane reduces the disincentive, so that latent desire is satisfied, until it reaches a new equilibrium.

Result: more people getting where they want to go.

Re: New LLM optimization technique slashes memory costs

#143
post #138

Earlier quoted context omitted.

I've been told my entire life that it's too late for nuclear, we should have been building them 20 years ago. I think now's fine, even if it takes time. these companies already buy a ton of power from renewable sources, and it's good to diversify - nuclear is a good backup to have.

The west tried building nuclear power 20 years ago. If it had delivered we would be building more now. It did not deliver. It is time to leave nuclear power to the past just like we have done with the steam engine. It had its heyday but better cheaper technology replaced it.

what does it mean that "the west tried" - was it a technical failure or was it that people didn't want it in their backyard? just because people hate something doesn't mean that they don't need it. children hate spinach.

Re: New LLM optimization technique slashes memory costs

#144
post #130

Earlier quoted context omitted.

This isn't like lossless compression. Both techniques involve throwing lots of information away, with the justification that doing so does not significantly affect the end result. The extent to which using both the techniques together will help will depend on how much overlap there is between the information each ends up discarding.

My joke was more along the lines of entropy. Entropy is information and you can't throw away all of it, otherwise you have nothing useful left.

Hence the idea to only throw away almost all of it.

Re: New LLM optimization technique slashes memory costs

#145

Earlier quoted context omitted.

They say it’s going to be online in 2028. Are you willing to bet that they won’t have 3 mile island operational by 2030?

> Constellation closed the adjacent but unconnected Unit 1 reactor in 2019 for economic reasons, but will bring it back to life after signing a 20-year power purchase agreement to supply Microsoft’s energy-hungry data centers, the company announced on Friday. The reactor they’re restarting was operational just five years ago. It’s not a fully decommissioned or melted down reactor and it’s likely all their licensing i…

The largest problem will be finding qualified and vetted personnel. All the people who worked at the plant when it closed five years ago had to find jobs elsewhere. Even though the plant was an important employer in Middletown, I don't know if those former employees will be willing the quit there current jobs to go back, especially if there is a risk the plant will just be shut down again when it once again becomes too expensive to operate.

Re: New LLM optimization technique slashes memory costs

#147
post #76

Earlier quoted context omitted.

Look, I agree that nuclear is difficult, but Google and Microsoft have publicly committed to those projects you’re mentioning. I don’t understand your dismissive tone that all of it is hogwash? This is one of those HN armchair comments.

My tone is because this is a simple predatory delay strategy. Tomorrow, tomorrow, I’ll decarbonize tomorrow. Instead of paying to buy wind and solar plants, which can go up today they are signing a meaningless agreement for the future . A PPA isn’t worth the paper it’s written on if the seller can’t produce electricity at the agreed upon price by the date required. Take Three Mile Island. It was closed in 2019 since…

The logic is pretty straightforward I’m not sure what your complaint is. They don’t need the power now, but they calculate that they’d need much more power in the future than non nuclear ways of power generation would be able to give them in the same timeframe.

The US is already adding record amount of solar and wind power to replace coal and natural gas plants. What makes you believe Microsoft can just buy more renewable energies? Did you privy to the terms and conditions of the PPA?

The alternative to Three Miles Island restart would be to add natural gas plants, or to buy renewable energy at higher price. I’m sure they have plan B.

Re: New LLM optimization technique slashes memory costs

#148

Earlier quoted context omitted.

More of lossy compression

Does lossy means you may see previous inputs change ?!

Only the model’s view, doesn’t have to be yours, just like you can participate in a long conversation without perfect memory that might in retrospect slightly differ from a recording.

Re: New LLM optimization technique slashes memory costs

#149
post #143

Earlier quoted context omitted.

The west tried building nuclear power 20 years ago. If it had delivered we would be building more now. It did not deliver. It is time to leave nuclear power to the past just like we have done with the steam engine. It had its heyday but better cheaper technology replaced it.

what does it mean that "the west tried" - was it a technical failure or was it that people didn't want it in their backyard? just because people hate something doesn't mean that they don't need it. children hate spinach.

There was talk of an ongoing nuclear renaissance in the early 2000s. [1]

American companies and utilities announced 30 reactors. Britain announced ~14.

We went ahead and started construction on 7 reactors in Vogtle, Virgil C. Summer, Flamanville, Olkiluoto and Hanhikivi to rekindle the industry. We didn't believe renewables would cut it.

The end result of what we broke ground on is 3 cancelled reactors, 3 reactors which entered commercial operation in the 2020s and 1 still under construction.

The rest are in different states of trouble with financing with only Hinkley Point C slowly moving forward.

In the meantime renewables went from barely existing to dominating new capacity (TWh) in the energy sector.

Today renewables make up 2/3rds of global investment in the energy sector.

The failure of nuclear power is that it is horrifically expensive and the timelines are insane compared to the competition.

Steam locomotives technically work, but are like nuclear power uncompetitive.

Lately nuclear power has caught the imagination of conservative politicians as a method to delay the renewable disruption of the fossil industry and have an answer to climate change.

When their plans, like in Australia, get presented they don’t care the slightest about nuclear power and it is only a method to prolong the life of the coal and gas assets.

[1]: https://en.wikipedia.org/wiki/Nuclear_renaissance_in_the_Uni...

Re: New LLM optimization technique slashes memory costs

#150
post #48

Is it possible that after 3-4 years of performance optimizations, both algorithmic and in hardware efficiency, it will turn out that we didn’t really need all of the nuclear plants we’re currently in the process of setting up to satisfy the power demands of AI data centers?

Nobody is building nuclear power plants for data centres. A few people have signed some paperwork saying that they would buy electricity from new nuclear plants if they could deliver it at a certain price, a price mind you that has not been done before. Others are trying to restart an existing reactor at three mile island (a thing that has never been done before, and likely won't be done now since the reactor was shu…

> Nobody is building nuclear power plants for data centres

Argentina just announced they're building nuclear plants for AI.

Post reply on HN