This seems so ludicrous to me when all we really need is a way to share a resource reference across sites. Like “I need react 18.1 on this page, and the SHA should be abcdefghi “. If you don’t have it, I can give it to you from my server, or you can follow this link to a CDN, but the resource itself can be deduplicated based on the hashed contents instead of the URI. Why isn’t this a thing when basically everything uses frameworks nowadays? This shared dictionary seems like a more obtuse and roundabout way to solve these. If there was caching by hashes, browsers could even preload the latest versions of new libraries before any sites even referenced them.
Compression efficiency with shared dictionaries in Chrome
61–70 of 81 posts
Re: Compression efficiency with shared dictionaries in Chrome
#62This seems so ludicrous to me when all we really need is a way to share a resource reference across sites. Like “I need react 18.1 on this page, and the SHA should be abcdefghi “. If you don’t have it, I can give it to you from my server, or you can follow this link to a CDN, but the resource itself can be deduplicated based on the hashed contents instead of the URI. Why isn’t this a thing when basically everything u…
Privacy issues.
You can use the presence of an item in the cache to correlate visits between sites.
Re: Compression efficiency with shared dictionaries in Chrome
#63This seems so ludicrous to me when all we really need is a way to share a resource reference across sites. Like “I need react 18.1 on this page, and the SHA should be abcdefghi “. If you don’t have it, I can give it to you from my server, or you can follow this link to a CDN, but the resource itself can be deduplicated based on the hashed contents instead of the URI. Why isn’t this a thing when basically everything u…
One potential issue is tracking. By sharing caches across websites it becomes possible to use timing attacks to track different users. This is why browsers are working to isolate caches per site: https://developer.chrome.com/blog/http-cache-partitioning
Re: Compression efficiency with shared dictionaries in Chrome
#64Earlier quoted context omitted.
Ya. Where is accept-encoding: zstandard-d-es2024 Where it encodes js files with a known dictionary that is ideal for es2024
And here’s one tuned for react, and one for svelte…
That wouldn’t make sense as it would be the user agent (aka your browser) that implements these shared dictionaries and they wouldn’t be able to add non-standard shared dictionaries for libs like react.
If they could do that then they might as well preload the cache with all common libs like react from well known cdn urls.
Re: Compression efficiency with shared dictionaries in Chrome
#65Even putting aside CORS because I don’t even want to think about how this plays well with requests to another (tracking?) domain, this still doesn’t seem worth it. The explicit use case seems to be that it basically tells the server when you last visited the site based on which dictionary you have and then it gives you the moral equivalent of a delta update. Except, most browsers are working hard to expire data of th…
The dictionaries are partitioned by document and origin so a "tracking" domain will only be able to correlate requests within a given document origin and not across sites.
They are also cleared any time cookies are cleared and don't outlive what you can do today with cookies or Etags (and are using the most restrictive partitioning for that reason).
Re: Compression efficiency with shared dictionaries in Chrome
#66[dead]
Re: Compression efficiency with shared dictionaries in Chrome
#67> Available-Dictionary: :pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=: The savings are nice in the best case (like in TFA: switching from version 1.3.4 to 1.3.6 of a lib or whatever) but that Base64 encoded hash is not compressible and so this line basically adds 60+ bytes to the request. Kinda ouch for when it's going to be a miss?
Aren't misses pretty preventable?
The only reason the client is even asking is that the server sent them a header saying it might be beneficial to do so.
And the client definitely has the dictionary data. The only thing it needs is for the server to accommodate the request after leading it down that path in the first place.
I can picture how it could happen, though. If you didn't realize the cost, you might not try to prevent misses. Or you could have a configuration error like sending the header but forgetting to generate pre-compressed data in your build.
If this is a significant issue, a server could collect stats and generate warnings about situations where it's not pulling its weight. Or even automatically disable it if hit rates are terrible.
Re: Compression efficiency with shared dictionaries in Chrome
#68I agree with other comments concerned with fingerprinting, and it was my second thought reading through the article. But my first thought was how beneficial this could be for return visitors of a web app, and how it could similarly benefit related concerns, such as managing local caches for offline service workers. True, for documents (as is another comment’s focus) this is perhaps overkill. Although even there, a be…
Even in the "documents" case of the web there can be pretty significant savings if users tend to visit more than one page and they share some amount of structure.
On the first entry to the site you trigger the load of an external dictionary that contains the common parts of the HTML across the site and then future document loads can be delta-compressed against the dictionary, effectively delivering just the page-specific bits.
You need to amortize the cost of loading the dictionary across the other page loads but it's usually pretty compelling once users visit more than 2-3 pages.
Re: Compression efficiency with shared dictionaries in Chrome
#69Earlier quoted context omitted.
This is something game devs have been doing for decades. If you want to delta 1.0 to 1.1 that’s server side work you do once at deployment or build time, not on every request.
What happens when you release 1.2 and someone who has 1.0 visits? Do you generate a delta for every past version at build time?
You determine how far back you want to build deltas for. If you build deltas for the last 3 versions then you can send diffs for those users as well (as long as the dictionary hasn't expired). Or, you could just send the full response just like if dictionaries weren't supported.
Each site can decide what a "good" number of releases to build against based on typical release cycles and user visitation patterns.
Re: Compression efficiency with shared dictionaries in Chrome
#70The part I'm missing is how these dictionaries are created. Can I use the homepage to create my dictionary, so all other pages that share html are better efficiently compressed? How?
For a delta update of one version of a resource to the next, the resource itself is the dictionary (i.e. JS file).
For stand-alone dictionaries, the brotli code on github has a dictionary_generator that you can use to generate a dictionary. You give it a dictionary size and a bunch of input files and it will generate one. I have a version of it hosted on https://use-as-dictionary.com/ that you can pass up to 100 URLs to and it will generate a dictionary for you (using the brotli tool).