61–64 of 64 posts
It's extremely surprising me to that something like this - be it with gzip, lz4, or whatever - wasn't implemented at Cloudflare ages ago... And that the task to investigate fell to an intern. These are non-negligible numbers...
It's extremely surprising me to that something like this - be it with gzip, lz4, or whatever - wasn't implemented at Cloudflare ages ago... And that the task to investigate fell to an intern. These are non-negligible numbers...
I suspect it just hasn’t been a priority problem in the past. This could very well have been a premature optimization when compared to getting services up and running reliably.
Tangentially related, I applied a similar approach to compress the npm registry by over 90% on disk a few years back. Since most versions of a package are similar, you can delta encode them first and then compress them. The deltas are small and compress well as a collection with the original source files. For another use case, prior to compressing, I’ve applied a rolling hash to deterministically split the file. Then…
NPM packages doesn’t take up that much disk space for me with standard pnpm deduplication. The much bigger offender for me is Rust target dir: when people talk about vibe coding in Rust for performance, what they don’t mention (at least I’ve hardly ever seen it mentioned) is every trivial little tool pushes 1GB on disk and anything slightly nontrivial easily racks up multi-GB. Which hurts when you have lots of vibed…
It was part of the (now defunct) frea project: https://freajs.github.io/frea-website/
Earlier quoted context omitted.
I was informed many eons ago that one of the hot features of Rational was that when one user made a change that required recompiling the headers, those headers were uploaded to the server so that other users didn't have to recompile them. It had the nice effect that whoever caused the headers to need to be recompiled was the only one who had to pay the recompilation tax instead of everyone. You may be cleaving the pr…
>user made a change that required recompiling the headers, those headers were uploaded to the server so that other users didn't have to recompile them. This sounds like a security nightmare - there is a reason we have build servers (well there are many reasons, but build safety is one of them). But I have to admit the idea is clever.