Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Compression efficiency with shared dictionaries in Chrome

developer.chrome.com

Post reply on HN
Print view View original

41–50 of 81 posts

Re: Compression efficiency with shared dictionaries in Chrome

#41

Post by Scaevolus »

jgrahamc wrote: post #10
Scaevolus wrote: post #8

Earlier quoted context omitted.

Delta compression is a huge win for many applications, but it takes a careful hand to make it work well, and inevitably it gets deprecated as the engineers move on and bandwidth stops being a focus-- just like Railgun has been deprecated! https://blog.cloudflare.com/deprecating-railgun Maybe the basic problem is with how hard it is to find engineers passionate about performance AND compression?

I don't think your characterization of why Railgun was deprecated is accurate. From the blog post you link to: “I use Railgun for performance improvements.” Cloudflare has invested significantly in performance upgrades in the eight years since the last release of Railgun. This list is not comprehensive, but highlights some areas where performance can be significantly improved by adopting newer services relative to us…

Right, but isn't that part of the general trend of bandwidth becoming far cheaper in the last decade along with dynamic HTML becoming a smaller fraction of total transit?

A 95%+ reduction in bandwidth usage for dynamic server-side-rendered HTML is much less important in 2023 than 2013.

Re: Compression efficiency with shared dictionaries in Chrome

#42

Post by adgjlsfhk1 »

kevingadd wrote: post #34
adgjlsfhk1 wrote: post #33

Earlier quoted context omitted.

the fact that the default on the web is to ship something that needs a parser is very silly.

Depending on how you look at it, Java, .NET and WebAssembly all need parsers too, they just happen to be parsing a binary format instead of text.

yes, and technically so does x86, but there's a pretty big difference between formats where the data is normalized and expected to be correct and formats that are intended for users and need to do things like name resolution and error checking. Parsing a language made for machines is easy to do faster than you can read the data from ram, while parsing a high level language will often happen at <100mbps

Re: Compression efficiency with shared dictionaries in Chrome

#44

Post by hinkley »

saagarjha wrote: post #17

Even putting aside CORS because I don’t even want to think about how this plays well with requests to another (tracking?) domain, this still doesn’t seem worth it. The explicit use case seems to be that it basically tells the server when you last visited the site based on which dictionary you have and then it gives you the moral equivalent of a delta update. Except, most browsers are working hard to expire data of th…

I think I’d feel better with a fixed set of dictionaries based on a corpus that gets updated every year to match new patterns of traffic and specifications. Even if it’s less efficient.

Re: Compression efficiency with shared dictionaries in Chrome

#45

Post by hinkley »

wongarsu wrote: post #12
ComputerGuru wrote: post #3

This seems like a possibly huge user/browser fingerprint. Yes, CORS has been taken into account, but for massive touch surface origins (Google, Facebook, doubleclick, etc) this certainly has concerning ramifications. It’s also insanely complicated. All this effort, so many possible tuples of (shared dictionary, requested resource), none of which make sense to compress on-the-fly per-request, mean it’s specifically fo…

Don't want to set session cookies? Just provide user-specific compression dictionaries and use them as your session id! After all, how is the user supposed to notice they got a different dictionary than everyone else

Same problem with etags.

Re: Compression efficiency with shared dictionaries in Chrome

#46

Post by hinkley »

saagarjha wrote: post #18
matsemann wrote: post #16

How could a dictionary in the browsers that are pre-made with JS in mind fare? Aka instead of making a custom dictionary per resource I send to the user, I could say that "my scripts.js file uses the browser's built-in js-es2023-abc dictionary". So the browser's would have some dictionaries others could reuse. What's the savings on that approach vs a gziped file without any dictionary?

So Brotli already contains a dictionary that is trained on web traffic. I think the thing here is that Google wants to make sending YouTube 1.1 more efficient if you already have YouTube 1.0, but they can’t put YouTube 1.0 into the browser.

This is something game devs have been doing for decades.

If you want to delta 1.0 to 1.1 that’s server side work you do once at deployment or build time, not on every request.

Re: Compression efficiency with shared dictionaries in Chrome

#48

Post by kazinator »

With shared dictionaries you can compress everything down to under a byte.

Just put the to-be-compressed item into the shared dictionary, somehow distribute that to everyone, and then the compressed artifact consists of a reference to that item.

If the shared dictionary contains nothing else, it can just be a one-bit message whose meaning is "extract the one and only item out of the dictionary".

Re: Compression efficiency with shared dictionaries in Chrome

#49

Post by tarasglek »

TacticalCoder wrote: post #7

> Available-Dictionary: :pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=: The savings are nice in the best case (like in TFA: switching from version 1.3.4 to 1.3.6 of a lib or whatever) but that Base64 encoded hash is not compressible and so this line basically adds 60+ bytes to the request. Kinda ouch for when it's going to be a miss?

chrome team usually trials changes like this with extensive a/b testing via telemetry. Got to be a large overall win even with this.

Re: Compression efficiency with shared dictionaries in Chrome

#50

Post by Twirrim »

Scaevolus wrote: post #41
jgrahamc wrote: post #10

Earlier quoted context omitted.

I don't think your characterization of why Railgun was deprecated is accurate. From the blog post you link to: “I use Railgun for performance improvements.” Cloudflare has invested significantly in performance upgrades in the eight years since the last release of Railgun. This list is not comprehensive, but highlights some areas where performance can be significantly improved by adopting newer services relative to us…

Right, but isn't that part of the general trend of bandwidth becoming far cheaper in the last decade along with dynamic HTML becoming a smaller fraction of total transit? A 95%+ reduction in bandwidth usage for dynamic server-side-rendered HTML is much less important in 2023 than 2013.

Unless you're part of the large majority of people in the world on slower mobile networks. We keep designing and building for people with broadband / wifi, and missing out just how big the 3G / lousy latency markets are.
Post reply on HN