Live data from Hacker News

There's no reason for software to be slow anymore

danluu.com

281–290 of 530 posts

Re: There's no reason for software to be slow anymore

#281
post #255

Earlier quoted context omitted.

Yes, but it's also built by people who would rather tinker with AI than build MacOS apps. Intrinsic motivation is very hard to beat, especially in subtler areas like good UX or performant software.

OpenAI has 4,500 employees and a post-money valuation of $852 billion at its last funding round. You'd think they could get someone who specialises in writing MacOS apps to write their MacOS app.

They can probably hire them, and they'd get PIP'd for not jumping when management says jump.

Toxic corporate culture can go a long way to self-sabotage.

Re: There's no reason for software to be slow anymore

#282

Earlier quoted context omitted.

The refutation of your takeaway is autoresearch and similar. They can brute force novel optimizations (and generally achieve superhuman performance) when provided with an appropriate environment. Of course that doesn't mean they have a human level mental model and associated novel ideas. Brute force can be effective but remains entirely unsatisfying from an academic perspective.

True! How do you set up auto research loops? Are there any special tricks to it?

I don't know if it's the first and it certainly isn't state of the art at this point but I think karpathy/autoresearch is a quintessential starting point.

TBF as an earlier commenter noted it's "just" superoptimization using an LLM as the proposer so there's a lot of relevant prior art.

Re: There's no reason for software to be slow anymore

#283
post #255

Earlier quoted context omitted.

Yes, but it's also built by people who would rather tinker with AI than build MacOS apps. Intrinsic motivation is very hard to beat, especially in subtler areas like good UX or performant software.

OpenAI has 4,500 employees and a post-money valuation of $852 billion at its last funding round. You'd think they could get someone who specialises in writing MacOS apps to write their MacOS app.

No. They want the AI to do it.

Re: There's no reason for software to be slow anymore

#284

As long as there’s a trade off between experimentation and performance, software will always be slightly too slow.

And now the other aspect of the trade off is token budget allocation.

The author seems to be in a situation where you can burn as many token as you want. I don't know if that's a general situation.

(Even if you don't care about the environment impact of your computations, there is a dollar bill associated, and _someone_ cares very much about that.)

I can foresee a situation where devs will have to decide on how they allocate a fixed token budget - and then, faced with the option of "burning tokens to add a new feature requested by a customer for tomorrow's demo" or "burning tokens to maybe make the app faster in some edge case", the trade off will look a lot like the ones organisations made with human dev time.

This assumes that tokens are not going to get dramatically cheaper. I can't predict the future, but I don't see a path to that (or, are local models, and "a TPU in every machine" going to make the question irrelevant?).

I can definitely see a path were tokens get massively more expensive (let's meet six months after anthropic's IPO and check :D)

Re: There's no reason for software to be slow anymore

#285
The software model has to however be easy for agents to debug in a loop, then it works very well. I recently got Fable to take a desktop app (AzWriter[1] - screenshot Mac: https://imgur.com/31DBG04 + Linux https://imgur.com/1IavBvS) from about 150 Mb -> 80 - 90 Mb on a reasonably complex UI (around 40 pages of text, paginated, etc.), even beating KWrite (160Mb even on KDE, even though KWrite doesn't do pagination or complex text layout, was a surprise to me).

What was important for this was the fact that I can run JSON-defined e2e tests[2], also good to find frame-based leaks, stale-ID problems and general "program shows its using lots of memory in the task manager" (task managers are wildly inaccurate for this, as I found out).

So, I can just tell it "okay, loop this e2e.json test over and over again, use heaptrack, samply, find out why and exactly where it's slow, memory-hungry - find the section in the codebase, figure out a solution" and then let it run overnight. The biggest difficulty here is that many perf tools are still written for humans and that things like "how much memory are we using" is a wildly complex topic (lots of problems actually getting the correct number without over-allocation, memory allocator slack, OS-level page size, memory fragmentation, etc.).

But Fable was able to track down things like "LCD font hinting causes 90k allocations that are unnecessary", etc. etc. - which then also improve frame time, usually. Memory optimization + better perf pretty much go hand in hand (less allocations = more perf). I could track them down manually probably, but it would take way more time.

Having some basic understanding of data-oriented design, cache locality, memory tiering (L1/L2/L3/main RAM), does massively help with architecture decisions (e.g. Azul can use a single buffer for the entire DOM node list, in difference to normal browsers which do the more "object based" allocation model, which massively helped page breaking performance on html-to-pdf use cases[3]). Pure-functional style also helps (f(State) -> UI) because then it gets very easy to drill down exactly where things are slow and where caches are needed.

[1] https://azul.rs/ui/release/0.2.0#demos

[2] https://github.com/fschutt/azul/blob/master/e2e/css-anim-per...

[3] https://github.com/fschutt/azul/blob/master/layout/benches/f...

Note: UI Toolkit is still very WIP, docs + code are still slop, etc. - working on it. But I just wanted to add this.

Re: There's no reason for software to be slow anymore

#286
post #44

One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it. Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick. If your software has the affordance of a wait…

This is why my most recent website does its server-side rendering on the client. I'm not even being that sarcastic. We stream a subset of the user's data (1 mb at most) in the background and have a WASM client that has the exact same server views. On SPA events, the WASM blob intercepts a lot of requests and can instantly render. Makes a laggy connection feel pretty quick.

Why aren't those views just part of your SPA in the first place?

Re: There's no reason for software to be slow anymore

#287
post #44

One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it. Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick. If your software has the affordance of a wait…

This is an incomplete and quite superficial view of what is going on out there, in my opinion. I've worked on plenty of projects where the assumption was that since the round-trip to the server is going to take almost 100ms that'll dwarf anything that's going to happen on the server itself, justifying poor choices that lead to potentially adding a whopping 100ms onto that number. These numbers only get larger with a larger perceived "Nothing we can do about it" budget as well, programmers often feel justified in doing just about anything once round-trip time grows, not understanding that they're just adding to an already existing problem.

On top of that: Creating software that isn't outright wasteful in terms of performance isn't even hard, it's just a matter of not doing ridiculously dumb things. The problem I've observed in teams I've worked with is that the majority of programmers don't even know what the dumb things are, and wouldn't know how to even approach making something that's halfway fast.

Edit:

Unfortunately I think posts like these are only going to make the problem worse, because now people are going to ask for voodoo solutions to performance issues, when the answer to their problems was usually just "Maybe stop creating wasteful intermediate structures and just walk an array like a sane person" in 99% of cases. The first leg of any optimization journey in the average programmer's code will likely net tens or hundreds of times faster code, and that's actually all people were asking for.

The knowledge required to make those changes and understand them is fairly minimal, but the kinds of people who have to create spinners for webmail interfaces, have their application add 150ms on top of whatever round-trip you have for processing things counted in 5 digits, etc., have never bothered to even learn those things.

Re: There's no reason for software to be slow anymore

#289
post #44

One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it. Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick. If your software has the affordance of a wait…

I agree and see this as a side effect of a subtler thing. As people shall be replacable, software is designed and subtly mimics the organization's communication patterns. Features are cut up into the tiniest pieces with clear separation from the start (at least its claimed), and over time whatever change or feature seems overly complicated, won't be done or won't be done in a sane manner because its uneconomical.

You are starting to get software with ticket driven development layered around glue code for existing libraries. I see no problem with libraries, it is just the architecture and vertical understanding that leaves a lot of performance on the table, because refactoring insanity takes resources and a lot of talking, understanding and convincing to be done.

Doing this across big teams starts to have downsides. So one team doesn't have a particular use case implemented or understood it and does not want to support it and you run code to compensate for this.

Best example in a monolith case is oracle...

Re: There's no reason for software to be slow anymore

#290
Software speed is actually a binary thing. If it's as fast as the user then it's good. If it's not then it's bad. Think of it like a car - in theory different cars have different accelerations and vmaxes, but in practice if you can reach max allowed speed in your country (except Germany) and overtake all other cars you want to overtake, then you're happy; if you cannot then you're not.

Here in hackernews we have a bubble of people who are very smart so they process information at high speed in general, and have lots of experience with computers, so they process computer-related information with extra speed. We're like race drivers, squeezing every single bit of performance from the car, while majority of use cases for a car is being stuck in traffic on your way to work. Those people simply don't care about performance above 150km/h because they'll never reach such speeds so putting expensive, high-performance engines is a waste of resources.

The point I'm trying to make is that if software is fast enough for average user, and most users are fucking slow, there's no business need to further optimize it. You might keep arguing "but but but back in my days apps were instant and now they suck" but users simply do not care. This is why companies put shitty, time-wasting animations even if nothing is being done in the background.

Post reply on HN