Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There's no reason for software to be slow anymore

danluu.com

Post reply on HN
Print view View original

271–280 of 532 posts

Re: There's no reason for software to be slow anymore

#271

Post by josephg »

fc417fc802 wrote: post #198
josephg wrote: post #133

Earlier quoted context omitted.

> iykyk A story. I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a…

The refutation of your takeaway is autoresearch and similar. They can brute force novel optimizations (and generally achieve superhuman performance) when provided with an appropriate environment. Of course that doesn't mean they have a human level mental model and associated novel ideas. Brute force can be effective but remains entirely unsatisfying from an academic perspective.

True! How do you set up auto research loops? Are there any special tricks to it?

Re: There's no reason for software to be slow anymore

#272

Post by gritzko »

josephg wrote: post #133
mccoyb wrote: post #55

Here's this boiled down: > A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective This is superoptimization. We've known this since the 80s (Massalin, STOKE is more recent: https://github.com/StanfordPL/stoke ) The only novelty is that the proposer is now way better with LMs. Further, there's a large number of reasons for software writ…

> iykyk A story. I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a…

Hi Joseph,

A story for a story. I had my CRDT implementation in libdog, which does per-token CRDT weave/diff/merge over a DAG of git blobs. It was written by Claude 4.8 I believe, in several iterations. It was, as you may guess, a piece of neuro-slop that passed the tests by some miracle. Once I had some time to look into it, I used a trick: I supplied it with my article on Chronofolds and some helpful kicks in the butt. It implemented everything correctly on its k-th attempt, kConclusion: it sort of expands its context and prompt by association. It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why. (And how can you guess if you have not had the experience. We all grew with Commodores and suchlike. I recall Spectrums, Robotrons, and Poisks. My friend sits on exam committees, says the youth arrives flatlined after 3 years of GPT. His words. Whatever.)

Re: There's no reason for software to be slow anymore

#273

Post by moffkalast »

bdhdhduuyd wrote: post #264

For me the comparison has always been 3DsMax vs Blender. Same kind of software, same kind of features, but Blender is so much faster. Architectural decisions have always been important.

Ha, I'm always struggling with UI performance in Blender. Undo takes ages and freezes up everything, modifiers start lagging mouse movement as soon as you have more than a handful of polygons. I shudder to imagine what Max must be like.

Though I must admit you can have an absolute shit ton of ridiculously complex objects in a scene, and it will run smoothly if none of them is in edit mode which is at least fairly impressive.

Re: There's no reason for software to be slow anymore

#274

Post by aureate »

blfr wrote: post #255
chvid wrote: post #253

ChatGPT MacOSX is the only software that regularly crashes on my machine when its memory consumption for no apparent reason spins up towards 50 GB. And that software is build by some of the highest paid software engineers on the planet with full access to all the LLM compute in the world.

Yes, but it's also built by people who would rather tinker with AI than build MacOS apps. Intrinsic motivation is very hard to beat, especially in subtler areas like good UX or performant software.

OpenAI has 4,500 employees and a post-money valuation of $852 billion at its last funding round. You'd think they could get someone who specialises in writing MacOS apps to write their MacOS app.

Re: There's no reason for software to be slow anymore

#275

Post by hunterpayne »

malisper wrote: post #209
hunterpayne wrote: post #197

"LLMs are causing slow, bloated, code are going to eat crow once they re-write everything in super-optimized assembly." This person doesn't understand how to make efficient code. I can write code in almost any language (with a couple of exceptions) that outperforms "super-optimized assembly". Writing efficient code isn't about the language, and often isn't about the best algorithms either (but sometimes it is). Its a…

> This person doesn't understand how to make efficient code The author is one of the most knowledgeable people about performance there is

The author didn't write the quote.

Re: There's no reason for software to be slow anymore

#276

Post by zanderwohl »

ehnto wrote: post #44

One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it. Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick. If your software has the affordance of a wait…

This is why my most recent website does its server-side rendering on the client. I'm not even being that sarcastic. We stream a subset of the user's data (1 mb at most) in the background and have a WASM client that has the exact same server views. On SPA events, the WASM blob intercepts a lot of requests and can instantly render. Makes a laggy connection feel pretty quick.

Re: There's no reason for software to be slow anymore

#277

Post by zanderwohl »

energy123 wrote: post #141
ehnto wrote: post #44

One of the biggest causes of slowness is just waiting for web requests. The fact that so much software is either online or built using the same stack even if it isn't, puts all that software in this blocked/waiting state constantly while using it. Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick. If your software has the affordance of a wait…

Can't they use ML to predict where I'm going to click, and pre-cache the predicted page whenever the predicted button doesn't mutate important state? Or skip the difficult ML and have some basic rule of thumb that pre-caches frequent button clicks, using a markov chain, and conditioned on those pages being low bandwidth to pre-load.

McMaster Carr website prefetches _all_ links upon hover. Saves you like half a second in many cases.

Re: There's no reason for software to be slow anymore

#278

Post by tomveber »

hombre_fatal wrote: post #182
bensyverson wrote: post #92

Much like security, optimization is now a function of token spend—which in some ways means it's a function of care. The reason why software may continue to be slower or less secure than it could be is simply that no one cares enough to invest the time and money in improving it. Off-topic, but Dan, in the name of all that is holy, throw a `max-width: 40em;` on your blog!

That said, performance and security considerations can be automated by a reusable harness and I think within six months it will just be built in to things like claude code and codex the same way you might tack on a performance/security section to your project's AGENTS.md. A crude example is how, when building a large forum project years ago, I had a checklist I ran on every new http route: authentication, authorizati…

The checklist half automates cleanly, you can lint that every route calls authorize. It won't catch authorize being handed the wrong policy, which is the one that ships. And when I put a second model on review duty, the common failure is it agreeing with the first model's misreading of the spec, almost word for word. So I'd expect the forgetting class of bugs to mostly go away and the misread-the-spec class to sit exactly where it is.

Re: There's no reason for software to be slow anymore

#279

Post by fallingbananna »

fhub wrote: post #217
andersmurphy wrote: post #202

Earlier quoted context omitted.

I think a lot of the time when people say the network is slow. They really mean their backend is slow. With a fast backend ~1-5ms response times (not even that fast). Streaming compression over something like SSE to keep your response sub 1kb packet (roughly an ethernet MTU). With a push based model, pushing data to a user is half their RTT latency. They will only experience their full RTT on actions they trigger. No…

1-5ms response time is clearly hard for most real world endpoints.

I don't know how hard it is. But I can certainly say there is no business inscentive for it.

When it comes to improving performance by a few ms, or implementing a new feature, business people will always choose a new feature, unless the current performance is unbearably slow (we're talking regular 1.5s+ wait times for BE response).

And it's not even a modern problem, legacy software written 20 years ago has the same latency than most modern backends from my experience.

Post reply on HN