Engineering for Slow Internet • Hacker News Board

Re: Engineering for Slow Internet

#291

Post by armchair_expert » Fri, May 31, 2024, 6:10 PM UTC

camel-cdr wrote: post #62

danpalmer wrote: post #9

Having a lot of experience commuting on underground public transport (intermittent, congested), and living/working in Australia (remote), I can safely say that most services are terrible for people without "ideal" network conditions. On the London Underground it's particularly noticeable that most apps are terrible at handling network that comes and goes every ~2 minutes (between stops), and which takes ~15s to conne…

It's not only the services them self. I have a very slow mobile connection, and one thing that bothered me immensly is downloading images in the browser: How is it, than when I go to a .jpg url to view an image in the browser it takes way longer and sometimes times out, than hopping over to termux and running wget. I had this problem with both firefox and chrome based browsers. Note that even the wget download usuall…

You can try going into proxy settings and setting to "none" instead of autodetect. Also, the dns server used by the browser could be different (and slower).

Re: Engineering for Slow Internet

#292

Post by lukan » Fri, May 31, 2024, 6:11 PM UTC

qingcharles wrote: post #276

lukan wrote: post #137

Earlier quoted context omitted.

"I feel like some devs need to time-travel back to 2005 or something and develop for that era in order to learn how to build things nimbly." No need to invent time travel, just let them have a working retreat somewhere with only bad mobile connection for a few days.

Amen to this. And give them a mobile cell plan with 1GB of data per month. I've seen some web sites with 250MB payloads on the home page due to ads and pre-loading videos. I work with parolees who get free government cell phones and then burn through the 3GB/mo of data within three days. Then they can't apply for jobs, get bus times, rent a bike, top up their subway card, get directions.

Having an adblocker (firefox mobile works with uBlock origin) and completely deactivate loading of images and videos can get you quite far with limited connection.

Re: Engineering for Slow Internet

#293

Post by Aachen » Fri, May 31, 2024, 6:12 PM UTC

EdwardDiego wrote: post #60

I gave a fellow who'd just come off the ice a ride while he was hitchhiking, he was saying that the blog author was somewhat resented by others because his blog posts, as amazing as they are, tended to hog what limited bandwidth they already had while the images uploaded, but he was given priority because the administration realised the PR value of it. Which I thought ties into the discussion about slow internet nice…

I was wondering about the practicalities indeed. Not everyone knows when their OS or applications decided it is now a great time to update. You'll have a phone in your pocket that is unnecessarily using all the bandwidth it can get its hands on, or maybe you're using the phone but just don't realise that watching a 720p video, while barely functional, also means the person trying to load it after you cannot watch even 480p anymore (you might not notice because you've got buffer and they'll give up before their buffer is filled enough to start playing).

It seems as though there should be accounting so you at least know what % of traffic went to you in the last hour (and a reference value of bandwidth_available divided by connected_users so you know what % was your share if everyone had equal need of it), if not a system that deprioritises everyone unless you punched the button that says "yes I'm aware what bandwidth I'm using in the next [X≤24] hour(s) and actually need it, thank you" which'll set the QoS priority for your MAC/IP address to normal

Re: Engineering for Slow Internet

#294

Post by mike_hearn » Fri, May 31, 2024, 6:13 PM UTC

Sesse__ wrote: post #289

mike_hearn wrote: post #261

Earlier quoted context omitted.

I hadn't noticed and I guess this is part of the problem. Sorry this post turned into a bit of a rant but I wrote it now. When it was decided that HTML shouldn't be versioned anymore it became impossible for anyone who isn't a full time and very conscientious web dev to keep up. Versions are a signal, they say "pay attention please, here is a nice blog post telling you the most important things you need to know". If…

You may be interested in the Baseline initiative, then. ( https://web.dev/baseline/2024 )

That does look useful, thanks!

Re: Engineering for Slow Internet

#295

Post by Karrot_Kream » Fri, May 31, 2024, 6:20 PM UTC

alanpearce wrote: post #234

Karrot_Kream wrote: post #127

So I've hacked a lot on networking things over the years and have spent time getting my own "slow internet" cases working. Nothing as interesting as McMurdo by far but I've chatted and watched YouTube videos on international flights, trains through the middle of nowhere, crappy rural hotels, and through tunnels. If you have access/the power (since these tend to be power hungry) to a general-purpose computing device a…

This sounds fascinating. Do you have some articles describing your setup?

Hah no, but maybe I should. The reason I haven't is that most of my work is just glue code. I use yt-dlp to do Youtube downloads, make use of the Discord, Slack and Telegram APIs to access those services. I run NNCP and the bots in systemd units, though at this point I should probably bake all of these into a VM and just bring it up on whichever cloud instance I want to act as ingress. Cloud IPs stay static as long as the box itself stays up so you don't need to deal with DNS either. John Goerzen has a bunch of articles about using NNCP [1] that I do recommend interested folks look into but given the popularity of my post maybe I should write an article on my setup.

FWIW I think it's fine that major services do not work under these conditions, though I wish messaging apps did. Both WhatsApp and Telegram IME are well tuned for poor network conditions and do take a lot of these issues into account (a former WA engineer comments in this thread and you can see their attention to detail.) Complaining about these things a lot is sort of like eating out at restaurants and complaining at how much sodium and fat goes into the dishes: restaurants have to turn a profit and catering to niche dietary needs just isn't enough for them to survive. You can always cook at home and get the macros you want. But for you to "cook" your own software you need access to APIs and I'm glad Telegram, Slack, and Discord make this fairly easy. Youtube yt-dlp does the heavy lifting but I wish it were easier, at least for Premium subscribers, to access Youtube via API.

I find Slack to be the absolute worst offender networking-wise. I have no idea how, now that Slack is owned by Salesforce, the app experience can continue to be so crappy on network usage. It's obvious that management there does not prioritize the experience under non-ideal conditions in any way possible. Their app's usage of networks is almost shameful in how bad it is.

[1]: https://www.complete.org/nncp/

Re: Engineering for Slow Internet

#296

Post by YoshiRulz » Fri, May 31, 2024, 7:01 PM UTC

walterbell wrote: post #57

IETF draft proposal to extend HTTP for efficient state synchronization, which could improve UX on slow networks, https://news.ycombinator.com/item?id=40480016 The Braid Protocol allows multiple synchronization algorithms to interoperate over a common network protocol, which any synchronizer's network messages can be translated into.. The current Braid specification extends HTTP with two dimensions of synchronization:…

This sounds suspiciously like Matrix. Does is required buy-in from user agents or will it benefit existing browsers once implemented?

Re: Engineering for Slow Internet

#297

Post by NelsonMinar » Fri, May 31, 2024, 7:09 PM UTC

0xWTF wrote: post #2

I still think engineering for slow internet is really important, and massively under appreciated by most software developers, but ... LEO systems (like Starlink, especially StarLink) essentially solve the core problems now. I did an Arctic transit (Alaska to Norway) in September and October of 2023, and we could make FaceTime video calls from the ship, way above the Arctic Circle, despite cloud cover, being quite far…

Starlink has its own networking issues thanks to a lot of latency jitter and 0.5% or more packet loss. See the discussion from last month: https://news.ycombinator.com/item?id=40384959

The biggest issue for Starlink at the poles is, as you say, very sparse coverage. Also I suspect Starlink has to usually relay polar packets between satellites, not just a simple bent pipe relaying to a ground station.

Re: Engineering for Slow Internet

#298

Post by toast0 » Fri, May 31, 2024, 7:29 PM UTC

daveoc64 wrote: post #154

nicbou wrote: post #101

I travel a lot. Slow internet is pretty common. Also, right now my mobile data ran out and I'm capped at 8 kbps. Websites that are Just Text On A Page should load fast, but many don't. Hacker News is blazing fast, but Google's API docs never load. The worst problem is that most UIs fail to account for slow requests. Buttons feel broken. Things that really shouldn't need megabytes of data to load still take minutes to…

>Websites that are Just Text On A Page should load fast, but many don't. Hacker News is blazing fast, but Google's API docs never load. Things aren't always that simple. I'm in the UK, and my ping time to news.ycombinator.com is 147ms - presumably because it's not using a CDN and is hosted in the USA. cloud.google.com on the other hand has an 8ms ping time. So yes, Hacker News is a simple, low-JS page - but there can…

I have experience with having a webpage with a global audience, served from random US locations (east/west/texas, but no targetting) and pretty unbloated, to something served everywhere with geodns and twice the page weight... Load times were about the same before and after. If we could have kept the low bloat, I expect we would have seen a noticable improvement in load times (but it wasn't important enough to fight over)

Re: Engineering for Slow Internet

#299

Post by greenish_shores » Fri, May 31, 2024, 7:29 PM UTC

Scoundreller wrote: post #198

kylehotchkiss wrote: post #68

Earlier quoted context omitted.

WhatsApp has a massive audience in developing countries where it's normal for people to have slower internet and much slower devices. That perspective being so embedded in their development goals certainly has given WhatsApp good reason to be the leading messaging platform in many countries around the world

Even wealthy countries will have dead zones (Toronto subway until recently, and like 90% of the landmass), and at least in Canada, “running out of data” and just having none left (or it being extremely expensive) was relatively common until about the last year or two when things got competitive (finally!). Still have an entire territory where everything is satellite fed (Nunavut), including its capital.

Wow. I didn't knew that Nunavut is entirely satellite fed. That's very interesting to know, thanks. Do you have some more info, though? What kind of satellite - geostationary, LEO? Also which constellation has the most share of traffic from Nunavut?

Re: Engineering for Slow Internet

#300

Post by greenish_shores » Fri, May 31, 2024, 7:30 PM UTC

qingcharles wrote: post #284

nicbou wrote: post #104

Earlier quoted context omitted.

It works remarkably well when your phone runs out of data and you get capped at 8 kbps. Even voice calls work smoothly.

LOL 8kbps. Damn. That takes me back. I built the first version of one of the world's largest music streaming sites on a 9.6kbps connection. I was working from home (we had no offices yet) and my cable Internet got cut off. My only back up was a serial cable to a 2G Nokia 9000i. I had to re-encode a chunk of the music catalog at 8kbps so I could test it from home before I pushed the code to production. Psychoacoustic…

Nokia 9000i, so you had to work on CSD (which is usually billed per-minute, like dial-up), not even GPRS. How much did that cost you? :P

BTW, an interesting thing is that some/most carriers allow you to use CSD/HSCSD over 3G these days, and you can establish data CSD connection between two phone numbers, yielding essentially a dedicated L2 pipe which isn't routed over internet. Can have much lower latency and jitter if that's what you need. Some specialized telemetry is still using that, however as 3G is slowly getting phased out, it will probably have to change.