Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Will programmers write more efficient code during the memory shortage?

news.ycombinator.com

Post reply on HN
Print view View original

241–250 of 264 posts

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#241

Post by jcgrillo »

frumiousirc wrote: post #208

In particle physics, processing jobs face a limit of about 2GB/core. This is driven by the realities of "grid computing". The hardware in the computing facilities have approximately 2GB/core and jobs allocate based on number of core. This ratio has been fairly stable for a long time. Some facilities may have a bit more RAM per CPU core but 2GB/core is the general rule of thumb. I hazard a guess that this will not cha…

I remember reading somewhere about an HPC "rule of thumb" that was like 1GB RAM/1GHz of processing is about the right amount for most applications.. Don't recall where that came from or what the exact reasoning was. That kind of aligns with 2GB/core.. Was that a thing?

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#242

Post by mike_hearn »

jcgrillo wrote: post #232
mike_hearn wrote: post #180

Earlier quoted context omitted.

It's a bit more subtle than bad assumptions. The main place RAM usage is going to get optimized is on the server side, because the client's RAM is a tragedy of the commons. If you're reducing RAM usage while your competitor adds features then the extra RAM saved by your app will just be silently allocated to theirs, the device won't feel any different and the user will prefer your competitor. There is little incentiv…

> The main place RAM usage is going to get optimized is on the server side Agree, but it's going to be a long, hard, potentially ultimately unsuccessful uphill slog. I think the main obstacle is basically 2 decades of inertia. Approximately everyone believes that server side software needs to scale horizontally. So we optimize systems for this property--I can make it work better by just throwing more computers at it.…

FWIW I work part time on projects that are all about reducing RAM usage server side! RAM has been a big expense in the cloud for a long time. AI makes the situation critical but there's a lot of work done already. Consider that functions-as-a-service are basically all about shutting down idle servers to reclaim their RAM.

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#244

Post by jcgrillo »

mike_hearn wrote: post #242
jcgrillo wrote: post #232

Earlier quoted context omitted.

> The main place RAM usage is going to get optimized is on the server side Agree, but it's going to be a long, hard, potentially ultimately unsuccessful uphill slog. I think the main obstacle is basically 2 decades of inertia. Approximately everyone believes that server side software needs to scale horizontally. So we optimize systems for this property--I can make it work better by just throwing more computers at it.…

FWIW I work part time on projects that are all about reducing RAM usage server side! RAM has been a big expense in the cloud for a long time. AI makes the situation critical but there's a lot of work done already. Consider that functions-as-a-service are basically all about shutting down idle servers to reclaim their RAM.

FaaS are really interesting, because the tradeoffs seem "jagged" (to use a word I learned recently). Like, IIUC this is more or less what's happening when a cloud function is invoked:

1. Some event triggers the function (HTTP request hits load balancer or whatever)

2. A VM is loaded from NVMe (possibly on another computer, and transferred over the network) into RAM

3. That VM boots up

4. The function code runs

5. The VM is terminated

Naively, this probably takes a lot more RAM (not to mention all the other computing resources) than to just have another HTTP handler sitting idle on some computer somewhere.

So this is only a reasonable choice if doing this intensive computation for a short amount of time, and using a large amount of resources for that short amount of time, is somehow better than doing a much smaller amount (maybe none at all) of computing over a longer amount of time. It's kind of hard to tell what the actual conditions are that merit one solution over the other. The other layer is the cloud pricing model, but that's a whole other can of worms..

I have yet to encounter a system that didn't have at least some computers (or cloud instances) that always need to be running. Those computers are almost always a bit overprovisioned (not using 100% of their CPU or RAM or IO all the time). So it has always seemed to me that instead of running a cloud function, it would be cheaper in terms of compute resources to execute the code from step 4 above on one of those machines.

Obviously we have cloud functions for a reason, people like them and find them useful. But is there ever a regime where they're optimal?

EDIT: I guess I've implicitly assumed the code in step 4 is doing a small amount of work. Maybe there's a goldilocks zone of work-per-invocation where this model is more optimal? But then of course there's always the standard executor pool model, and that way you don't have to spin up a VM per task invocation...

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#245

Post by dredmorbius »

mike_hearn wrote: post #180
jakub_g wrote: post #168

I don't believe so do for two reasons: - everyone assumes their program / website is the only thing running at the machine at a given time, and dev machines are always more powerful than user machines - it's not really lack of advanced data structures and algorithms that result in the bloat most of the time but the fact that programs and websites are delivered by large teams, there are dozens of submodules that are o…

It's a bit more subtle than bad assumptions. The main place RAM usage is going to get optimized is on the server side, because the client's RAM is a tragedy of the commons. If you're reducing RAM usage while your competitor adds features then the extra RAM saved by your app will just be silently allocated to theirs, the device won't feel any different and the user will prefer your competitor. There is little incentiv…

If you're reducing RAM usage while your competitor...

This is true so long as client-side (desktop, mobile) memory management does not penalise high memory use.

I'm already taking the approach of killing off my largest browser processes regularly, and need to look at more targeted ways of managing memory. I'd really like to see the parent browser process as a lightweight manager over subprocesses such that it can persist (rather than leaking multiple GB of RAM over the course of days), to the point my entire user session falls over with stunning regularity.

I have a shell script (currently triggered manually, hopefully subject to further refinement) which kills off the ten top browser processes. I'll often run that in a shell loop of 10--20 iterations. It barely keeps things manageable, and system hangs/reboots are still far more common than I'd like.

The way to change behaviours is to change costs. This is where OS devs have a choice before them, and application and remote service / SaaS devs and project managers might eventually start feeling the pain.

One reason I favour HN over numerous other options is that the site doesn't absolutely pig out my browser session(s).

And that said, if anyone has tips on both revealing and managing memory usage in Firefox, I'm quite receptive.

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#246

Post by xp84 »

mike_hearn wrote: post #185
xp84 wrote: post #52

Earlier quoted context omitted.

> bundling Chromium to make a chat app is the problem. Precisely. And all that extra resource wastage is completely free! (paid by your customers). Perhaps if there were any big software companies who were so iconoclastic as to write fast software and avoid wasteful patterns like using Electron, pressure to do better could be felt, but every company that ships software[1] behaves the same so if anyone tries out compe…

You're forgetting about Apple and that entire ecosystem. Android, too. Not much Electron to be found there. Both Android and iOS were heavily optimized for low memory environments back at the start.

First-party and lovingly-crafted indie apps do use native technologies, and often care about performance, but it feels like everything else uses various bloated cross-platform frameworks.

The fact that nothing on my modern day iPhone feels any faster than my iPhone 4 did, when the current phones are 10x more powerful tells me that developers consistently spend >= 100% of the yearly Moore's Law dividend on framework bloat, spyware, and useless animations. If we could run iOS 6 and its matching apps on modern hardware, and set animation duration to 0 like jailbreaks used to allow, imagine how fast our phones would be!

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#249

Post by vengefulduck »

tristanj wrote: post #203
BearOso wrote: post #37

Earlier quoted context omitted.

We're in this situation because of laziness. I doubt doubling down on it is the solution.

Programmers will just /goal the LLM to make the code more memory efficient. Then let it run it run overnight.

And it will spend 1 million tokens finding the most memory efficient program:

int main(){}

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#250

Post by thewhitetulip »

TiredOfLife wrote: post #199
inigyou wrote: post #74

Earlier quoted context omitted.

Is there a single example of a successful LLM project apart from Claude Code?

Bun, Ghostty

Migration projects probably won't be the best fit here

An existing project has a well defined code base, test cases etc

If llms aren't able to migrate a good project then they wouldn't be of any use for general purpose programming

Post reply on HN