Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There's no reason for software to be slow anymore

danluu.com

Post reply on HN
Print view View original

51–60 of 530 posts

Re: There's no reason for software to be slow anymore

#51

Post by devin »

Lerc wrote: post #26

Giving an LLM a program that does a well defined thing correctly and asking it to make a faster version and a crapload of tests to show that it performs exactly the same task is a great way to burn through a bunch of tokens and getting a faster program in return. I currently have one running working on a multi path problem trying to make an A* heuristic that produces the same or better score for search time, paths cr…

I would be genuinely interested to see how it evolves the same program over a large number of generations. Given a whole lot of software is more a collection of programs rather than a singular "program", I have to imagine that the outcomes would be vastly different in shape when dealing with smaller targeted, articulable-in-a-few-paragraphs programs vs many-years-of-business-domain layered programs.

Re: There's no reason for software to be slow anymore

#53

Post by hsn915 »

ehnto wrote: post #44

One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it. Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick. If your software has the affordance of a wait…

It's not very hard to engineer software with these two constraints at the same time:

* Must feel very responsive * Network requests can take up to 500ms end to end

Re: There's no reason for software to be slow anymore

#55

Post by mccoyb »

Here's this boiled down:

> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective

This is superoptimization. We've known this since the 80s (Massalin, STOKE is more recent: https://github.com/StanfordPL/stoke) The only novelty is that the proposer is now way better with LMs.

Further, there's a large number of reasons for software written by agents to be slow:

- LMs still don't do data or hardware-oriented design well out of the box, and therefore if you're engaging in any sort of serious novel work, beyond porting an extremely well-understood program with extremely well-understood workloads, you're going to be spending hours tracking down bad allocation decisions (c.f. why TigerBeetle doesn't use agents), which are often the root of evil (before you'd reach for anything further)

- The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.

The reality is to get performant code (quickly) out of an agent, you need to know how to write performant code (and you need to know how to surface the information that you'd use to create a verifier for such a thing to the agent), which 99% of developers do not know in 2026.

Sure, agents can teach you how to do this -- but it's one of these things where iykyk.

Experience: I've poured 10s of billions of tokens into Zig with the best agents and I have the time and space to try these things.

If you want to start learning the discipline, I'd recommend matklad's + TigerBeetle blog -- as well as hardware-oriented design.

Re: There's no reason for software to be slow anymore

#56

Post by 0x20cowboy »

champagnepapi wrote: post #25
Grombobulous wrote: post #16

Earlier quoted context omitted.

I’m a lot more optimistic now that RAM is scarce. A lot of focus will be on maximizing software performance. See: iOS 27. It’s faster than the previous version, even on very old phones. I also think AI will contribute to removing a lot of the tedium surrounding optimization.

People these days are saying “don’t look at the code”. People are shipping all sorts of weird architectures, non-performance code, etc. I wish that the ram scarcity would drive more performant software however I just think that the way people are shipping software currently will not lead to this

> People are shipping all sorts of weird architectures, non-performance code, etc.

And they weren’t before AI?

Re: There's no reason for software to be slow anymore

#57

Post by devin »

y1n0 wrote: post #29
newsomix9xl wrote: post #14

One article said secure software was here because of AI, this one says it can now be performant. Yet when I ask for code it writes, by default, both slow and insecure code that mostly works. Kinda. As I try to get AI to rewrite it into more secure, less bloated and optimized code is when it starts to randomly crash. Then I read articles about how AI is "moving too fast" and cry.

It's all about the test suite. The test suite becomes an executable specification, and the better the spec, the better the results you can get from AI.

The very first thing many people did with AI is start calling the automated tests it writes good enough to capture the desired behavior. Writing tests isn't a whole lot of fun for most, so now the thing that it is supposed to help ensure we're not evolving our software into piles of trash is, in fact, one of the most neglected parts of vibed out codebases.

Re: There's no reason for software to be slow anymore

#58

Post by w-ll »

datsci_est_2015 wrote: post #48

Feel like this is the equivalent of a traffic engineer standing at the grand opening of the 5th lane for the highway saying there’s no reason for traffic to be slow anymore. That is, there’s a misunderstanding of why software (traffic) was slow in the first place, and it has nothing to do with our ability to generate code (number of lanes), even if that code is “high quality”.

you dont mesaure the quality of a plane by how much it weighs. but yea i think everything is rot to the core. app to os if just wackness that nobody cared about.

Re: There's no reason for software to be slow anymore

#59

Post by thorian1828i03 »

moron4hire wrote: post #42
y1n0 wrote: post #29

Earlier quoted context omitted.

It's all about the test suite. The test suite becomes an executable specification, and the better the spec, the better the results you can get from AI.

If you have such a test suite then you really don't need AI to write the code for you.

Writing a benchmark test is about 100x to 1000x easier than writing optimizations. A benchmark test can be as simple as:

    func Benchmark(b *testing.B) {
      for range b.N {
        runCodeUnderBenchmark()
      }
    }
Actually optimizing the runCodeUnderBenchmark() function is far more difficult.
Post reply on HN