Engineering for Slow Internet • Hacker News Board

Re: Engineering for Slow Internet

#281

Post by miyuru » Fri, May 31, 2024, 5:25 PM UTC

palata wrote: post #272

miyuru wrote: post #141

Wonder if the author tried BitTorrent. Despite its usage issues, its built with reliability in mind and its a great way to transfer large files.

How do you use WhatsApp over BitTorrent? And how do you update your macOS over BitTorrent? The author clearly says that downloaders elaborate enough to deal with slow connections (e.g. "download in a browser") were fine. The problem is that modern apps don't let you download the file the way you want, they just expect you to have a fast internet connection.

No, I just meant if he tried to get a large file(Linux ISO) with BitTorrent. which should be reliable in theory.

Bittorent has webseeds support, which can use apples direct CDN urls to create a torrent file to download. archive.org still uses this technique and AWS S3 used to do this when they had torrent support.

There a website that do just that, it creates a torrent file from any direct weburl.

https://www.urlhash.com/

Re: Engineering for Slow Internet

#282

Post by chipdart » Fri, May 31, 2024, 5:28 PM UTC

Nevolihs wrote: post #176

chipdart wrote: post #29

Earlier quoted context omitted.

> Low bandwith, high latency connections need to be part of the regular testing of software. One size does not fit all. It would be a waste of time and effort to architect (or redesign) an app just because a residual subset of potential users might find themselves on a boat in the middle of the Pacific. Let's keep things in perspective: some projects even skip testing WebApps on more than one browser because they dee…

Websites regularly break because I don't have perfect network coverage on my phone every single day. In a lot of places, I don't even have decent reception. This in Germany in and around a major city. Why do you think this only applies to people on a boat?

> Websites regularly break because I don't have perfect network coverage on my phone every single day.

Indeed, that's true. However, the number of users that go through similar experiences are quite low and even those who do are always a F5 away from circumventing that issue.

I repeat: even supporting a browser other than the latest N releases of Chrome is a hard sell to some companies. Typically the test matrix is limited to N versions of Chrome and the latest release of Safari when Apple products are supported. If budgets don't stretch even to cover the basics, of course that even rarer edge cases such as a user accessing a service through a crappy network will be far from the list of concerns.

Re: Engineering for Slow Internet

#283

Post by AdamH12113 » Fri, May 31, 2024, 5:37 PM UTC

It's interesting that these are exactly the sort of conditions that the internet protocols were designed for. A typical suite of internet software from the 90s would handle easily.

One key difference is that client software was installed locally, which (in modern terms) decouples UI from content. As the article points out, the actual data you're dealing with is often measured in bytes. An email reader or AOL Instant Messenger would only have to deal with that data (plus headers and basic login) instead of having to download an entire web app. And since the protocols didn't change often, there was no need to update software every few weeks (or months, or even years).

Another key difference, which is less relevant today, is that more data came from servers on the local network. Email and Usenet were both designed to do bulk data transfer between servers and then let users download their individual data off of their local server. As I recall, email servers can spend several days making delivery attempts before giving up.

Re: Engineering for Slow Internet

#284

Post by qingcharles » Fri, May 31, 2024, 5:44 PM UTC

nicbou wrote: post #104

kylehotchkiss wrote: post #68

Earlier quoted context omitted.

WhatsApp has a massive audience in developing countries where it's normal for people to have slower internet and much slower devices. That perspective being so embedded in their development goals certainly has given WhatsApp good reason to be the leading messaging platform in many countries around the world

It works remarkably well when your phone runs out of data and you get capped at 8 kbps. Even voice calls work smoothly.

LOL 8kbps. Damn. That takes me back. I built the first version of one of the world's largest music streaming sites on a 9.6kbps connection.

I was working from home (we had no offices yet) and my cable Internet got cut off. My only back up was a serial cable to a 2G Nokia 9000i. I had to re-encode a chunk of the music catalog at 8kbps so I could test it from home before I pushed the code to production.

Psychoacoustic compression is a miracle.

Re: Engineering for Slow Internet

#285

Post by chefandy » Fri, May 31, 2024, 5:46 PM UTC

bobbob1921 wrote: post #259

nicoburns wrote: post #241

Earlier quoted context omitted.

> which is obviously weighed down by the libraries, but largely is as heavy or light as you make it One thing which surprised me at a recent job was that even what I consider to be a large bundle size (2MB) didn't have much of an effect on page load time. I was going to look into bundle splitting (because that included things like a charting library that was only used in a small subsection of the app). But in the end…

My experience agrees with this comment – I’m not sure why web browsers seem to frequently get hung up on only some Http requests at times, unrelated to the actual network conditions. Ie: in the browser the HTTP request is timing out or in a blocked state and hasn’t even reached the network layer when this occurs. (Not sure if I should be pointing the finger here at the browser or the underlying OS). However, when tes…

If if the request itself hasn't reached the network layer but is having a networky feeling hang, I'd look into DNS. It's network dependent but handled by the system so it wouldn't show up in your web app requests. I'm sure there's a way to profile this directly but unless I had to do it all the time I'd probably just fire up wireshark.

Re: Engineering for Slow Internet

#286

Post by qingcharles » Fri, May 31, 2024, 5:47 PM UTC

Tade0 wrote: post #155

nicbou wrote: post #101

I travel a lot. Slow internet is pretty common. Also, right now my mobile data ran out and I'm capped at 8 kbps. Websites that are Just Text On A Page should load fast, but many don't. Hacker News is blazing fast, but Google's API docs never load. The worst problem is that most UIs fail to account for slow requests. Buttons feel broken. Things that really shouldn't need megabytes of data to load still take minutes to…

One additional benefit of static sites, which I learned the hard way, is that you're mostly immune to attacks. I have a domain that's currently marked as "dangerous" because I didn't use the latest version of Wordpress.

I had a client that I set up with a static site generator. Sadly the client changed their FTP password to something insecure and someone FTP'd in and added a tiny piece of code to every HTML file!

Re: Engineering for Slow Internet

#287

Post by qingcharles » Fri, May 31, 2024, 5:48 PM UTC

luuurker wrote: post #208

Those of us working on apps, websites, etc, need to remember that there are lots of people out there that are not connected to the fast Wi-Fi or fibre connections we have. Here in the UK, some networks started shutting down 3G. Some have 2G as a low energy fall back, but we're supposed to use 4G/5G now. The problem is that 4G is not available everywhere yet, some areas until recently only had good 3G signal. So I've…

Here in the USA a great number of networks will drop back to 2G when their data plan runs out. And most poor people are on really low data limits, so they spend most of the month on 2G.

Try using Google Maps to get around on 2G :(

Re: Engineering for Slow Internet

#288

Post by jmbwell » Fri, May 31, 2024, 5:51 PM UTC

qingcharles wrote: post #276

lukan wrote: post #137

Earlier quoted context omitted.

"I feel like some devs need to time-travel back to 2005 or something and develop for that era in order to learn how to build things nimbly." No need to invent time travel, just let them have a working retreat somewhere with only bad mobile connection for a few days.

Amen to this. And give them a mobile cell plan with 1GB of data per month. I've seen some web sites with 250MB payloads on the home page due to ads and pre-loading videos. I work with parolees who get free government cell phones and then burn through the 3GB/mo of data within three days. Then they can't apply for jobs, get bus times, rent a bike, top up their subway card, get directions.

"But all the cheap front-end talent is in thick client frameworks, telemetry indicates most revenue conversions are from users on 5G, our MVP works for 80% of our target user base, and all we need to do is make back our VC's investment plus enough to cash out on our IPO exit strategy, plus other reasons not to care" — self-identified serial entrepreneur, probably

Re: Engineering for Slow Internet

#289

Post by Sesse__ » Fri, May 31, 2024, 5:58 PM UTC

mike_hearn wrote: post #261

Sesse__ wrote: post #237

Earlier quoted context omitted.

> CSS is a very inefficient way to represent styling logic which is why web devs write dialects like sass that are more compact, but browsers don't adopt it. I don't know exactly which features you are referring to, but you may have noticed that CSS has adopted native nesting, very similarly to Sass, but few sites actually use it. Functions and mixins are similar compactness/convenience topics being worked on by the…

I hadn't noticed and I guess this is part of the problem. Sorry this post turned into a bit of a rant but I wrote it now. When it was decided that HTML shouldn't be versioned anymore it became impossible for anyone who isn't a full time and very conscientious web dev to keep up. Versions are a signal, they say "pay attention please, here is a nice blog post telling you the most important things you need to know". If…

You may be interested in the Baseline initiative, then. (https://web.dev/baseline/2024)

Re: Engineering for Slow Internet

#290

Post by chefandy » Fri, May 31, 2024, 5:59 PM UTC

nicoburns wrote: post #241

chefandy wrote: post #225

Earlier quoted context omitted.

Yep. In SPAs with good architecture, you only need to load the page once, which is obviously weighed down by the libraries, but largely is as heavy or light as you make it. Everything else should be super minimal API calls. It's especially useful in data-focused apps that require a lot of small interactions. Imagine implementing something like spreadsheet functionality using forms and requests and no JavaScript, as o…

> which is obviously weighed down by the libraries, but largely is as heavy or light as you make it One thing which surprised me at a recent job was that even what I consider to be a large bundle size (2MB) didn't have much of an effect on page load time. I was going to look into bundle splitting (because that included things like a charting library that was only used in a small subsection of the app). But in the end…

In many cases, like satellite Internet access or spotty mobile service, for sure. But if you have low bandwidth but fast response times, that 2mb is murder and the big pile o requests is NBD.If you have slow response times but good throughput, the 2MB is NBD but the requests are murder.

An extreme and outdated example, but back when cable modems first became available, online FPS players were astonished to see how much better the ping times were for many dial up players. If you were downloading a floppy disk of information, the cable modem user would obviously blow them away, but their round trip time sucked!

Like if you're on a totally reliable but low throughput LTE connection, the requests are NBD but the download is terrible. If you're on spotty 5g service, it's probably the opposite. If you're on, like, a heavily deprioritized MVNO with a slower device, they both super suck.

It's not like optimization is free though, which is why it's important to have a solid UX research phase to get data on who is going to use it, and what their use case is.