Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There's no reason for software to be slow anymore

danluu.com

Post reply on HN
Print view View original

261–270 of 532 posts

Re: There's no reason for software to be slow anymore

#261

Post by dandellion »

4lx87 wrote: post #28

Jevon's paradox implies we're getting more fast software, and more slow software. Sturgeon’s law implies the ratio will remain the same. 90% of what ships will be slow, buggy, slop. Just as it was before LLMs.

I think this is the most accurate take on this thread. I've written super optimised code in assembly and SIMD. But only for contests never at work, because I've never had an employer that cared to optimise things that far, they just wanted cheap software that worked just good enough to get the job done. Even in places where they actually cared about quality would usually prefer adding a couple features over optimising things that much.

Re: There's no reason for software to be slow anymore

#262

Post by chvid »

blfr wrote: post #255
chvid wrote: post #253

ChatGPT MacOSX is the only software that regularly crashes on my machine when its memory consumption for no apparent reason spins up towards 50 GB. And that software is build by some of the highest paid software engineers on the planet with full access to all the LLM compute in the world.

Yes, but it's also built by people who would rather tinker with AI than build MacOS apps. Intrinsic motivation is very hard to beat, especially in subtler areas like good UX or performant software.

I think the software out of the big AI labs: OpenAI's ChatGPT MacOSX app, Anthropic Claude Code - is a sign of the future to come.

Big, feature rich, built in quick iterations - but also, in particular if you look under the hood, of extremely poor quality if measured by traditional software engineering standards (code structure as exemplified by the leaked Claude Code source code, resource usage, "buggyness" etc.).

Re: There's no reason for software to be slow anymore

#263

Post by arvigeus »

jjcm wrote: post #70

This speaks to me. I've been running an autoresearch loop the past couple of days to improve the load time of my various projects' frontends. I've been really, really impressed with how effective this is. I went from a 4s load on simulated slow 4g to ~750ms: https://image.non.io/speedup-graphs.webp Side by side vid of the results: https://video.non.io/speedups.mp4 This was for https://non.io , which is something I ha…

Autoresearch is crazy good! Closest thing to “Make this app fast!” we have now.

Re: There's no reason for software to be slow anymore

#265

Post by DeepSeaTortoise »

energy123 wrote: post #141
ehnto wrote: post #44

One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it. Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick. If your software has the affordance of a wait…

Can't they use ML to predict where I'm going to click, and pre-cache the predicted page whenever the predicted button doesn't mutate important state? Or skip the difficult ML and have some basic rule of thumb that pre-caches frequent button clicks, using a markov chain, and conditioned on those pages being low bandwidth to pre-load.

Oh god, don't give them ideas. All ML is in-cloud AI now. I dread the day everything around my mouse movements needs to get tokenized and vibed into the ClosedAI cloud before my buttons start working again.

Re: There's no reason for software to be slow anymore

#266

Post by NoMoreNicksLeft »

alightsoul wrote: post #185
ehnto wrote: post #44

One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it. Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick. If your software has the affordance of a wait…

In my experience the main driver of latency is not ping time, but how long the server takes to process the request.

How long the server takes to process the 42 requests ahead of you in line, or possibly how long the 318 poorly architected microservices take.

Re: There's no reason for software to be slow anymore

#267

Post by bambax »

ehnto wrote: post #44

One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it. Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick. If your software has the affordance of a wait…

UI blocking can make sense in some situations; otherwise things "happen" suddenly that are unexpected.

I'm making a simple plugin for Gimp that sends the active layer to a model with a prompt; on Macs by default the UI waits for the request to come back or timeout; on PCs it doesn't, so the modified layer appears unexpectedly. The Mac experience is better IMHO and I will replicate it in the PC version rather than the other way around.

Re: There's no reason for software to be slow anymore

#269

Post by troupo »

andersmurphy wrote: post #202
alightsoul wrote: post #185

Earlier quoted context omitted.

In my experience the main driver of latency is not ping time, but how long the server takes to process the request.

I think a lot of the time when people say the network is slow. They really mean their backend is slow. With a fast backend ~1-5ms response times (not even that fast). Streaming compression over something like SSE to keep your response sub 1kb packet (roughly an ethernet MTU). With a push based model, pushing data to a user is half their RTT latency. They will only experience their full RTT on actions they trigger. No…

Reddit's reaponses are rarely above 400ms. And yet their frontend routinely takes several seconds to render that response

Re: There's no reason for software to be slow anymore

#270

Post by josephg »

zem wrote: post #245
josephg wrote: post #133

Earlier quoted context omitted.

> iykyk A story. I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a…

yep, i think the real LLM superpower is knowing that something has been done before and having access to the code that did it. so much of even novel software includes bits and pieces that have well-optimised existing solutions, and the bot knows those solutions a lot better than i do, and can even pattern match them from the general shape of the problem.

Yes. It’s also excellent at reading large codebases and putting together a picture of what’s going on. I’ve been using it a lot lately to brief me on projects and design decisions. “Look at these two projects. They both solve task X. Write a report about their similarities and differences, and the tradeoffs as a result.” And then I ask followup questions. Saves a ton of time.
Post reply on HN