Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Will programmers write more efficient code during the memory shortage?

news.ycombinator.com

Post reply on HN
Print view View original

231–240 of 264 posts

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#231

Post by sgarland »

ponector wrote: post #226
sgarland wrote: post #193

Earlier quoted context omitted.

> the fact that programs and websites are delivered by large teams, there are dozens of submodules that are often loaded even when not needed Precisely this. We had an incident once where a CSV had a postal code field be interpreted as an integer by pandas, which of course results in stripping any leading 0s. After looking at what the code needed to do, I asked why they were using pandas in the first place, as it was…

It's called resume-driven-development. You can't add stdlib into your resume while you can pandas. If there are no guardrails, loose quality control and nobody cares about performance - why should they replace current pandas with stdlib?

I naïvely assumed that avoiding being paged was motivation enough, but experience has disabused me of that notion. Time after time, “we’ll fix one more thing on our hacky, bespoke bullshit” has won out over “we will use boring, pre-existing solutions.” My favorite example was a team who built a load balancer / health checker for Postgres in NodeJS. Despite causing roughly one incident per month, I was never able to convince anyone that they should abandon their special baby.

I honestly don’t understand how it’s even helpful to put shit like that on your résumé. If I read that in an interview, my immediate question would be “why did you build this instead of using HAProxy / AWS NLB / etc.?”

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#232

Post by jcgrillo »

mike_hearn wrote: post #180
jakub_g wrote: post #168

I don't believe so do for two reasons: - everyone assumes their program / website is the only thing running at the machine at a given time, and dev machines are always more powerful than user machines - it's not really lack of advanced data structures and algorithms that result in the bloat most of the time but the fact that programs and websites are delivered by large teams, there are dozens of submodules that are o…

It's a bit more subtle than bad assumptions. The main place RAM usage is going to get optimized is on the server side, because the client's RAM is a tragedy of the commons. If you're reducing RAM usage while your competitor adds features then the extra RAM saved by your app will just be silently allocated to theirs, the device won't feel any different and the user will prefer your competitor. There is little incentiv…

> The main place RAM usage is going to get optimized is on the server side

Agree, but it's going to be a long, hard, potentially ultimately unsuccessful uphill slog. I think the main obstacle is basically 2 decades of inertia. Approximately everyone believes that server side software needs to scale horizontally. So we optimize systems for this property--I can make it work better by just throwing more computers at it. This was a lot more important ca. 2004 (MapReduce) than it is ca. 2026. For resiliency and redundancy, how many nodes does your service actually need? If you're running more than that number, do you have some kind of justification as to why? This is the mindset we need to try to move towards, but it's very different from how we currently do it.

EDIT: Another good question to ask, which we don't do enough: "does it really need to be a separate service?" Using data in RAM that's already been allocated is probably better than allocating more and shipping data over the network...

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#233

Post by jcgrillo »

Machine telemetry data (logs, metrics, traces) are shipped around as highly redundant utf-8 strings. I'm sure there are plenty of other examples of 2+ orders of magnitude improvements that aren't even low hanging fruit, they're basically groundfall. But picking them up may nonetheless require quite a bit of effort, and a major reevaluation of corporate values. Maybe the RAM crunch will be sufficient incentive? Idk.

EDIT: It's also possible that the era of hypergrowth might end. Most of the world's addressable eyeballs are already being monetized. There's no new coal seam to mine. If tech becomes more about doing the same with less, and less about explosively growing at all costs, we'll probably see more efficient systems. And fewer people building them. I think this kind of larger systemic change would be more likely to drive efficiency improvements than the (probably more temporary) RAM crunch.

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#234

Post by sgarland »

DontchaKnowit wrote: post #229
sgarland wrote: post #193

Earlier quoted context omitted.

> the fact that programs and websites are delivered by large teams, there are dozens of submodules that are often loaded even when not needed Precisely this. We had an incident once where a CSV had a postal code field be interpreted as an integer by pandas, which of course results in stripping any leading 0s. After looking at what the code needed to do, I asked why they were using pandas in the first place, as it was…

Reason number 500 why I hate pandas with a passion. Just had basically the exact same thing happen at work

I have no problem with pandas (though polars is immensely faster, for those cases where it matters), just as I have no problem with numpy; what I dislike is people reaching for them for the most trivial of things that are already solved by stdlib. It’s even more irritating when they cite performance as the necessary reason, but they’re just passing data in a loop, negating nearly all the potential gains.

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#238

Post by jcgrillo »

thewebguyd wrote: post #78
devmor wrote: post #77

Earlier quoted context omitted.

“Stop doing stupid stuff” is usually the biggest performance gain in commercial development. We do stupid stuff as a stopgap to meet a deadline and then stupid stuff stays until it starts being a problem.

There's nothing more permanent than a temporary fix. If deadlines are causing those stopgaps, the aggressive deadlines are the root problem.

I swapped the engine on my lawn mower ~3yr ago because it no longer had any compression and was very hard to start. The new engine has an electric starter. I didn't have time to fabricate a proper battery tray so I ratchet strapped a 2x4 to the gas tank and strapped the battery to that. It stayed that way until last month when I finally had time to weld up a battery tray.

Sometimes when some janky p.o.s. solution works well enough, it truly is good enough.

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#239

Post by devin »

dosisking wrote: post #104
devin wrote: post #6

Earlier quoted context omitted.

I can refer you to some documentation on the topic. ;)

I'm pretty sure the person who you are responding to already knows and does not need your documentation. But most 'programmers' are just Python programmers and not real programmers.

1. It was a joke about pointers.

2. Cut out the macho gatekeeping nonsense around what makes a "real" programmer. It's tiresome.

Re: Ask HN: Will programmers write more efficient code during the memory shortage?

#240

Post by dang »

Lionga wrote: post #109

[flagged]

We've banned this account for repeatedly breaking the site guidelines with snarky, shallow, and aggressive comments across many threads. We can't allow that here because it destroys what the site is supposed to be for.

If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html.

Post reply on HN