I’m currently on a moving cruise ship in the Mediterranean with a starlink connection. A latency of 300-500 ms seems to be normal. Although bandwidth is tolerable at 2-4 mbps during the day with hundreds of passengers using it. At night it gets better. But latency can still be frustrating.
Engineering for Slow Internet
271–280 of 406 posts
Re: Engineering for Slow Internet
#272Wonder if the author tried BitTorrent. Despite its usage issues, its built with reliability in mind and its a great way to transfer large files.
The author clearly says that downloaders elaborate enough to deal with slow connections (e.g. "download in a browser") were fine. The problem is that modern apps don't let you download the file the way you want, they just expect you to have a fast internet connection.
Re: Engineering for Slow Internet
#273Re: Engineering for Slow Internet
#274Earlier quoted context omitted.
And the insistence on pushing everything into JS instead of just serving the content. So you’ve got to wait for the skeleton to dl, then the JS, which’ll take its sweet time, just to then(usually blindly) make half a dozen _more_requests back out, to grab JSON, which it’ll then convert into html and eventually show you. Eventually.
Well grabbing json isn't that bad. I made a CLI for ultimateguitar ( https://packages.debian.org/sid/ultimateultimateguitar ) that works by grabbing the json :D
Re: Engineering for Slow Internet
#275This is why we need more incremental rendering[1] (or "streaming"). This pattern become somewhat of a lost art in the era of SPAs — it's been possible since HTTP/1.1 via chunked transfer encoding, allowing servers to start sending a response without knowing the total length. With this technique, the server can break down a page load into smaller chunks of UI, and progressively stream smaller parts of the UI to the cl…
Re: Engineering for Slow Internet
#276A lot of this resonates. I'm not in Antartica, I'm in Beijing, but still struggle with the internet. Being behind the great firewall means using creative approaches. VPNs only sometimes work, and each leaves a signature that the firewall's hueristics and ML can eventually catch onto. Even state-mandated ones are 'gently' limited at times of political sensitivity. It all ends up meaning that, even if I get a connectio…
"I feel like some devs need to time-travel back to 2005 or something and develop for that era in order to learn how to build things nimbly." No need to invent time travel, just let them have a working retreat somewhere with only bad mobile connection for a few days.
I've seen some web sites with 250MB payloads on the home page due to ads and pre-loading videos.
I work with parolees who get free government cell phones and then burn through the 3GB/mo of data within three days. Then they can't apply for jobs, get bus times, rent a bike, top up their subway card, get directions.
Re: Engineering for Slow Internet
#277The other problem is that nobody in the software industry gives a damn. Everyone wants to make shiny apps with the last shiny tech. Try to mention optimizing for slow hardware/Internet and look at the face of your colleagues, behind their brand new M3.
I worked in a company with some remote colleagues in Africa. There were projects that they could literally not build, because it would require downloading tens of GB of docker crap multiple times a week for no apparent reason. The solution was to not have those colleagues work on those projects. Nobody even considered that maybe there was something to fix somewhere.
Re: Engineering for Slow Internet
#278Take Home Depot for example. Loading their website in a mobile browser is soooooooooo slow. The rendering is atrocious, with elements jumping all over the place. You click on one thing and it ends up activating a completely different element, then you have to wait for whatever you just clicked to load and jump all over the place again. Very frustrating! Inside their stores is even worse! I asked for help locating an item from one of their workers one day and they pull up their in-store app. That too was slower than molasses and janky, so we ended up standing there for several minutes just chatting waiting for it to load.
Re: Engineering for Slow Internet
#279Re: Engineering for Slow Internet
#280Earlier quoted context omitted.
Well, I'm working right now so let me check our daily "productivity" sites (with an adblocker installed): - Google Mail: Inbox is ~18MB (~6MB Compressed). Of that, 2.5MB is CSS (!) and the rest is mostly JS - Google Calendar: 30% lower, but more or less the same proportions - Confluence: Home is ~32MB (~5MB Comp.). There's easily 20MB of Javascript and at least 5MB of JSON. - Jira: Home is ~35MB (~7MB compressed). I…
I sometimes wish I could spare the time just to tear into something like that Slack number and figure out what it is all doing in there. Javascript should even generally be fairly efficient in terms of bytes/capability. Run a basic minimizer on it and compress it and you should be looking at something approaching optimal for what is being done. For instance, a variable reference can amortize down to less than one byt…
You talk about minification. The JS isn't minified much. Variable names are single letter, but property names and more aren't renamed, formatting isn't removed. I guess the minifier can't touch property names because it doesn't know what might get turned into JSON or not.
There's plenty of logging and span tracing strings as well. Lots of code like this:
_n.meta = {
name: "createThunk",
key: "createThunkaddEphemeralMessageSideEffectHandler",
description: "addEphemeralMessageSideEffect side effect handler"
};
The JS is completely generic. In many places there are if statements that branch on all languages Slack was translated into. I see checks in there for whether localStorage exists, even though the browser told the server what version it is when the page was loaded. There are many checks and branches for experiments, whether the company is in trial mode, whether the code is executing in Electron, whether this is GovSlack. These combinations could have been compiled server side to a more minimal set of modules but perhaps it's too hard to do that with their JS setup.Everything appears compiled using a coroutines framework, which adds some bloat. Not sure why they aren't using native async/await but maybe it's related to not being specialized based on execution environment.
Shooting from the hip, the learnings I'd take from this are:
1. There's a ton of low hanging fruit. A language toolchain that was more static and had more insight into what was being done where could minify much more aggressively.
2. Frameworks that could compile and optimize with way more server-side constants would strip away a lot of stuff.
3. Encoding logs/span labels as message numbers+interpolated strings would help a lot. Of course the code has to be debuggable but hopefully, not on every single user's computer.
4. Demand loading of features could surely be more aggressive.
But Slack is very popular and successful without all that, so they're probably right not to over-focus on this stuff. Especially for corporate users on corporate networks does anyone really care? Their competition is Teams after all.