Live data from Hacker News

Engineering for Slow Internet

brr.fyi

361–370 of 406 posts

Re: Engineering for Slow Internet

#361

Earlier quoted context omitted.

Yes - a thousand yeses. I once got asked "what was a life changing company/product" and my answer was WhatsApp - to slightly bemused looks. WhatsApp connected the world for free. Obviously they weren't the first to try but when my (very globally distributed family) picked up WhatsApp in '09/'10 we knew we were onto something different. Being able to stay in touch with my brother half way across the world in realtime…

> Email felt clunky and oddly formal - email clients don't feel "chatty". Now (in 2024) have you tried Delta Chat?

No I haven't - it looks interesting thanks for sharing.

I did once think whether a client could abstract over IMAP to build to build a WhatsApp-like UI/UX so clearly other people have thought the same.

Although I will confess I'm no longer really looking for a new chat experience...

How is the latency is for Delta Chat?

Re: Engineering for Slow Internet

#362

Earlier quoted context omitted.

Amen to this. And give them a mobile cell plan with 1GB of data per month. I've seen some web sites with 250MB payloads on the home page due to ads and pre-loading videos. I work with parolees who get free government cell phones and then burn through the 3GB/mo of data within three days. Then they can't apply for jobs, get bus times, rent a bike, top up their subway card, get directions.

This is probably a deliberate decision on the part of the government. A lot of the justice system is designed to keep people in prison.

That makes little sense. If it was a deliberate plan, it would be much more effective (and cheaper) to not provide the cellphones and data plan in the first place.

Re: Engineering for Slow Internet

#363
post #292

Earlier quoted context omitted.

Amen to this. And give them a mobile cell plan with 1GB of data per month. I've seen some web sites with 250MB payloads on the home page due to ads and pre-loading videos. I work with parolees who get free government cell phones and then burn through the 3GB/mo of data within three days. Then they can't apply for jobs, get bus times, rent a bike, top up their subway card, get directions.

Having an adblocker (firefox mobile works with uBlock origin) and completely deactivate loading of images and videos can get you quite far with limited connection.

And disabling JavaScript

Re: Engineering for Slow Internet

#364
I've been looking into how to survive low bandwidth/high latency/frequent disconnecting internet, because we may need to save money at home and temporarily switch to an unlimited 2G speed (64kbit) cell data plan as our primary internet for a few months. (Red Pocket's $7/mo. 1GB GSMA plan paid annually.) Not only is Red Pocket notoriously bad at limiting both bandwidth and are horrible with latency, we also live far from the closest cell tower, so our reception is also poor and frequently disconnects. A one-two-three punch for a bad internet experience.

This is probably about as close to experiencing an Antarctic data satellite connection within the continental US as is possible, and would be even worse than a 56k modem; While the bandwidth is roughly the same, Red Pocket users often report latencies in excess of 1000+ms, whereas 56k modem latency was typically under 200ms. The frequent disconnects (because we're far from the tower) would simulate Antarctic conditions, as well.

So how do internet-connected people living in 2024 survive such an atrocity?

First, we would leverage SSH and CLI tools as much as possible. Brow.sh and Carbonyl browser for web and Mutt for email. Mosh.sh to keep the SSH connection alive and reconnect automatically on reconnect--even if your IP changes. (Mosh is incredible.) Some terminals have a zoom feature (ctrl++ or - in kitty) which may help if some web browser item is just too difficult to see in its blocky resolution. See also the Carbonyl --bitmap and --zoom flags: https://github.com/fathyb/carbonyl/releases/tag/v0.0.3

The youtube-dl command can fetch tiny versions of a video on a server (even audio-only if we don't care about the video portion such as with long-form interviews), then transcode them on the server to the smallest-tolerable bitrate. rsync them home overnight and watch them the next day. rsync can retry on failure: rsync --partial --progress --compress-choice=STR, --zc=zlib --compress-level=9 --rsh=ssh user@host:remote_file local_file

Edit: On second thought, rsync doesn't perform well in high-latency environments due to TCP acks, but something like uftp which uses UDP should do better, and offers resume. https://stackoverflow.com/questions/38014501/what-is-the-fas... https://uftp-multicast.sourceforge.net/ https://sourceforge.net/p/uftp-multicast/discussion/general/...

Needs a remote Linux server, but ServerHunter has servers adequate for the task for less than $3/mo. Focus on RAM; Most any plan has sufficient hard drive space and speed, sufficient bandwidth and CPU for what you're doing. You're not serving large content to many demanding customers, and in my personal testing I have found that even 1GB RAM should be enough--but obviously more is better. And many companies only charge for data egress--which should be smaller than ingress. Data ingress to the server is usually free, and when you're downloading large amounts of data to the server, then compressing them, egress is much smaller. Choose companies that have a domain age of at least 1 year old; I've had a fly-by-night company take my money and run. (PayPal reimbursed me.)

To download large files, try first compressing them; Although, depending on the original file, compression can either give you no advantage or even _increase_ the file size. Try different compression methods; bzip2, zpaq, xz, lzma, etc. ls -l the before and after to see if it helped any.

Also, we'd rely on phones apps for things like Gmail/Facebook/Messenger/WhatsApp/Telegram/Twitter/Discord/etc. This saves always redownloading JavaScript for each visit. While there are Windows/Linux desktop clients for these, they usually are only thin layer over a browser, which means they still download JavaScript and large images on every use. But a phone app has everything it needs all the time--until the next app update, of course. Can temporarily disable app updates, but I wouldn't do that for long. The good part about app updates is they can tolerate slow links and frequent disconnects. And we often take our phones with us to high-bandwidth locations, so we can do app updates there and disable them over cell data. The article said on some days Antarctica has better bandwidth availability, so you could halt updates until conditions improve.

Same with Windows or Linux OS auto-updates; Disable them until bandwidth conditions improve. (But you're probably already doing that.) Edit: After reading the article again, yes they definitely are doing that, and are still having problems. Bummer.

Installing Bluestacks is an option, for the above reason. Using phone apps on the desktop that don't need to download JavaScript on every use could bridge the gap somewhat. Disable its auto-updates as well.

Gmail can be set up for IMAP or POP3. Try both; POP should give a better experience, but by default it removes messages on the server, so look for that setting to disable it if you want to keep them there. I believe you can disable downloading attachments and images automatically, which probably works better for IMAP? Not sure if POP can handle this. Doing it this way means you don't need to load the web interface any time you check mail, and you'll only be reading the text content of emails. It should therefore work through Mutt as well.

Then there's browser tricks for local rendering: Disable graphic images and disable media autoplay. uBlock Origin to reduce overall download size. Turn off JavaScript until truly needed. Pain in the butt, but it works.

In Windows, set your network connection to metered. This should help. (And back in the US, we can set our phones to low bandwidth mode--but that would only help users on cell networks; I'm sure the south pole lacks cell reception.)

You're not going to be gaming with large downloads and patches, nor will you be competing with low-latency gamers around the globe. But not all games need these. Play against the computer instead of other people. Play only pre-downloaded or DVD-based games. Or go back to the 90s and set up a LAN party and play against one another in the same room. We love Unreal Tournament 1999 GOTY; Plays blisteringly fast on any computer, is cheap to buy on DVD or Steam, has no in-game paid-for upgrades, has oodles and oodles of maps for download, and gameplay still feels fresh even 25 years later. Card games and offline phone games are king in low bandwidth environments.

You're probably not going to make many audio or video calls, so learn to send mp4s or mp3s to others back and forth by email. Use the lowest-tolerable bitrate. A 6Kbps OPUS/32Kbps AV1 video is about 5KB/sec and is clear enough to follow along, and would certainly be acceptable for a video message. A 0.5Kbps OPUS-encrypted audio file is surprisingly understandable, if not very enjoyable. But to get the message across, it works. A 6Kbps OPUS audio sounds like telephone quality to my ears. https://heavydeck.net/post/64k-is-enough-for-video/

Remember that 24-64kbit audio streams are still a thing. Search low bandwidth streaming radio. Sort by bitrate: https://directory.shoutcast.com/ https://directory.shoutcast.com/?q=299_Talk_297

I don't know if podcast apps have a low-bandwidth option, but it's worth investigating. Apparently the Player FM Pro app compresses pods on their servers before sending.

There are a few text-only news sites: https://blog.wturrell.co.uk/text-only-news-websites/

RSS feeds can fetch news over a slow link, and they're still a thing: https://rss.feedspot.com/world_news_rss_feeds/

Finally, there's the possibility of running a remote desktop session: Something like RDP over SSH, VNC over SSH, NoMachine's NX, TeamViewer, etc. Can give clearer detail than brow.sh and can sometimes be faster than browsing locally. I tested it by intentionally limiting my desktop's speed to 5KB/sec and while it wasn't fun, it got the job done. Took about ten seconds to paint a page on a 1GB VM, but that can often be faster than rendering a JavaScript-heavy page locally. Seems to be a good compromise for certain situations when the rest of the options fail you.

Typing is too laggy (5 seconds before characters appeared) but one way around that is to type into a file via SSH then cat the file on the server, paste that into the web form. Or write your notes to a file on your local system and copy+paste that directly in. I've found that even a server with 1GB RAM is sufficient for even this task, as long as you're only looking at one web page at a time. No multiple tabs. I ran XFCE4 on Ubuntu 22.04 and Google's official Chrome package. Resized the connection to 1024x768, which I consider to be the minimum for useful browsing. Changed the quality settings to lowest, color depth to lowest, etc. Disabled audio and printing.

To read large pages offline, print to PDF on the server inside RDP/VNC and download that with rsync.

The combination of all of these should help us get through a season of needing to save money, and it may very well save the day in Antarctica, as well.

Re: Engineering for Slow Internet

#365
Big tech probably loses millions of customers from developing countries because of their bloated frontend engineering. Not everyone has an M3 with gigabit fiber.

My favorite setup when working remotely with slow internet is the following :

- Build tasks & large data operations run on my home server via SSH

- Markdown notes, Notion being unusable

- 100% CLI apps, which are usually the lightest and most efficient

Re: Engineering for Slow Internet

#366
post #26

I had a similar experience as the author on a boat in the south pacific. Starlink was available but often wasn't used because of its high power usage (60+ watts). So we got local SIM cards instead which provided 4G internet in some locations and EDGE (2G) in others. EDGE by itself isn't too bad on paper - you get a couple dozen kilobits per second. In reality, it was much worse. I ran into apps with short timeouts th…

The South Pacific should be very sunny. I guess that you didn't have enough solar panels to provide 60+ watts. I am genuinely surprised. And "local SIM cards" implies that you set foot on (is)lands to buy said SIM cards. Where did you only get 2G in the 2020s? I cannot believe any of this is still left in the South Pacific.

In the Marquesas and Tuamotus, you don't see a lot of 4G reception, no matter what Vini's pretty map claims.

Re: Sunny - there's quite a bit of cloud cover and other devices onboard like the water maker and fridge (more important than Starlink!) also need a lot of power.

Re: Engineering for Slow Internet

#367
post #362

Earlier quoted context omitted.

This is probably a deliberate decision on the part of the government. A lot of the justice system is designed to keep people in prison.

That makes little sense. If it was a deliberate plan, it would be much more effective (and cheaper) to not provide the cellphones and data plan in the first place.

You see this all the time. Need to keep up appearances but the actually implementation is poor.

Re: Engineering for Slow Internet

#368
post #333

Earlier quoted context omitted.

What's odd is that I have the exact opposite: I find it uncomfortable to use megabytes of libraries with hundreds of dependencies when I can spend 30 seconds and discover the html5 native way of doing something. But my applications look butt ugly. They're fast and privacy preserving, but also it requires a special skill, from my point of view, to make them look good (be it design skills or knowing how to use these fr…

The "html5 native way of doing things" is not necessarily inherently fast (or reliable) without scripting tbh.

I'm thinking of UI elements that are often half-implemented in JavaScript but also exist natively nowadays, such as sliders. What kind of thing are you thinking of?

Re: Engineering for Slow Internet

#369

Earlier quoted context omitted.

You're 100% right. uBlock Origin can reduce page weight by an astronomical amount.

uMatrix (unsupported but still works) reduces page weight and compute even more

If you enable the Advanced Features mode of Ublock Origin you get access to just about all the things uMatrix does

Re: Engineering for Slow Internet

#370
Give me a break.

The purpose of broadband is so that nothing has any latency when you're not on dial-up.

Except user-initiated mega downloads, and they occur at the maximum connection rate the hardware can handle. At bandwidth costs anybody can afford. Just like with dial-up, why settle for less instantaneous performance than the hardware can handle? Just to make it so lesser geeks can participate?

The hallmark of insufficient web "engineering" has always been the completeness of the failure to pre-deploy on dial-up.

Post reply on HN