Engineering for Slow Internet • Hacker News Board

Re: Engineering for Slow Internet

#191

Post by jrochkind1 » Fri, May 31, 2024, 1:25 PM UTC

padolsey wrote: post #43

A lot of this resonates. I'm not in Antartica, I'm in Beijing, but still struggle with the internet. Being behind the great firewall means using creative approaches. VPNs only sometimes work, and each leaves a signature that the firewall's hueristics and ML can eventually catch onto. Even state-mandated ones are 'gently' limited at times of political sensitivity. It all ends up meaning that, even if I get a connectio…

Chrome dev tools offer a "slow 3G" and a "fast 3G". Slow 3G?

With fresh cache on "slow 3G", my site _works_, but has 5-8 second page loads. Would you have consider that usable/sufficient, or pretty awful?

Re: Engineering for Slow Internet

#192

Post by austin-cheney » Fri, May 31, 2024, 1:26 PM UTC

70%+ of the web is putting text on screen and responding to user interactions, 25%+ is spyware and advertising, and the last 5% are cool applications. How complicated should that really be?

This is a good example of why I gave up a career as a JavaScript developer after 15 years. I got tired of fighting stupid, but even stupid woefully unqualified people need to make 6 figures spinning their wheels to justify their existence.

Re: Engineering for Slow Internet

#193

Post by nicoburns » Fri, May 31, 2024, 1:26 PM UTC

jiggawatts wrote: post #113

devjab wrote: post #85

Earlier quoted context omitted.

We design for slow internet, react is one of the better options for it with ssr, code splitting and http2 push, mixed in with more off-line friendly clients like Tauri. You can also deploy very near people if you work “on the edge”. I’m not necessarily disagreeing with your overall point, but modern JS is actually rather good at dealing with slow internet for server-client “applications”. It’s not necessarily easy to…

By far the lightest weight JS framework isn't React, it's no javascript at all . I regularly talk to developers who aren't even aware that this is an option.

In my experience page weight isn't usually the biggest issue. On unreliable connections you'll often get decent bandwidth when you can get through. It's applications that expect to be able to multiple HTTP requests sequentially and don't deal well with some succeeding and failing (or just network failures in general) that are the most problematic.

If I can retry a failed a network request that's fine. If I have to restart the entire flow when I get a failure that's unusable.

Re: Engineering for Slow Internet

#194

Post by jerf » Fri, May 31, 2024, 1:29 PM UTC

tecleandor wrote: post #134

mwcampbell wrote: post #122

Earlier quoted context omitted.

False dichotomy, with what is likely extreme hyperbole on the JS side. Are there actual sites that ship 20 MB, or even 5 MB or more, of frameworks? One can fit a lot of useful functionality in 100 KB or less of JS< especially minified and gzipped.

Well, I'm working right now so let me check our daily "productivity" sites (with an adblocker installed): - Google Mail: Inbox is ~18MB (~6MB Compressed). Of that, 2.5MB is CSS (!) and the rest is mostly JS - Google Calendar: 30% lower, but more or less the same proportions - Confluence: Home is ~32MB (~5MB Comp.). There's easily 20MB of Javascript and at least 5MB of JSON. - Jira: Home is ~35MB (~7MB compressed). I…

I sometimes wish I could spare the time just to tear into something like that Slack number and figure out what it is all doing in there.

Javascript should even generally be fairly efficient in terms of bytes/capability. Run a basic minimizer on it and compress it and you should be looking at something approaching optimal for what is being done. For instance, a variable reference can amortize down to less than one byte, unlike compiled code where it ends up 8 bytes (64 bits) at the drop of a hat. Imagine how much assembler "a.b=c.d(e)" can compile into to, in what is likely represented in less compressed space than a single 64-bit integer in a compiled language.

Yet it still seems like we need 3 megabytes of minified, compressed Javascript on the modern web just to clear our throats. It's kind of bizarre, really.

Re: Engineering for Slow Internet

#195

Post by password4321 » Fri, May 31, 2024, 1:33 PM UTC

It doesn't take much to slow down RDP over TCP (especially when port forwarding through SSH).

I did find mention of increasing the cache¹ and lowering the refresh rate to 4 fps² (avoiding unnecessary animations), but I still feel the need for a server-side QUIC proxy that is less pushy based on network conditions. There is a red team project that has the protocol parsed out in Python³ instead of all the ActiveX control clients.

¹ https://superuser.com/questions/13487/how-to-increase-perfor...

² https://learn.microsoft.com/en-us/troubleshoot/windows-serve...

³ https://github.com/GoSecure/pyrdp

Re: Engineering for Slow Internet

#196

Post by JeremyNT » Fri, May 31, 2024, 1:33 PM UTC

kylehotchkiss wrote: post #70

0xWTF wrote: post #2

I still think engineering for slow internet is really important, and massively under appreciated by most software developers, but ... LEO systems (like Starlink, especially StarLink) essentially solve the core problems now. I did an Arctic transit (Alaska to Norway) in September and October of 2023, and we could make FaceTime video calls from the ship, way above the Arctic Circle, despite cloud cover, being quite far…

Idealistic! I think a lot of countries are going to block starlink in the future by interfering with the signals, much like the success some countries are having interfering so heavily with GPS. Their governments won't want uncensored web, or an American company being the gateway to the internet. They'll maintain whatever territorial networks they have now and the speed question is still relevant. Also the number of…

Even if people want to / are allowed to, I'm trying to imagine how well starlink could plausibly function if 2 billion people switched from their sketchy terrestrial service to starlink.

As a luxury product used by a few people, maybe it "solves" the problem, but I don't think this is a very scalable solution.

Re: Engineering for Slow Internet

#197

Post by nicoburns » Fri, May 31, 2024, 1:34 PM UTC

rcxdude wrote: post #140

mike_hearn wrote: post #116

Earlier quoted context omitted.

If you're behind an overloaded geosynchronous satellite then no JS at all just moves the pain around. At least once it's loaded a JS-heavy app will respond to most mouse clicks and scrolls quickly. If there's no JS then every single click will go back to the server and reload the entire page, even if all that's needed is to open a small popup or reload a single word of text.

This makes perfect sense in theory and yet it's the opposite of my experience in practice. I don't know how, but SPA websites are pretty much always much more laggy than just plain HTML, even if there are a lot of page loads.

It often is that way, but it's not for technical reasons. They're just poorly written. A lot of apps are written by inexperienced teams under time pressure and that's what you're seeing. Such teams are unlikely to choose plain server-side rendering because it's not the trendy thing to do. But SPAs absolutely can be done well. For simple apps (HN is a good example) you won't get too much benefit, but for more highly interactive apps it's a much better experience than going via the server every time (setting filters on a shopping website would be a good example).

Re: Engineering for Slow Internet

#198

Post by Scoundreller » Fri, May 31, 2024, 1:35 PM UTC

kylehotchkiss wrote: post #68

danpalmer wrote: post #9

Having a lot of experience commuting on underground public transport (intermittent, congested), and living/working in Australia (remote), I can safely say that most services are terrible for people without "ideal" network conditions. On the London Underground it's particularly noticeable that most apps are terrible at handling network that comes and goes every ~2 minutes (between stops), and which takes ~15s to conne…

WhatsApp has a massive audience in developing countries where it's normal for people to have slower internet and much slower devices. That perspective being so embedded in their development goals certainly has given WhatsApp good reason to be the leading messaging platform in many countries around the world

Even wealthy countries will have dead zones (Toronto subway until recently, and like 90% of the landmass), and at least in Canada, “running out of data” and just having none left (or it being extremely expensive) was relatively common until about the last year or two when things got competitive (finally!).

Still have an entire territory where everything is satellite fed (Nunavut), including its capital.

Re: Engineering for Slow Internet

#199

Post by galangalalgol » Fri, May 31, 2024, 1:36 PM UTC

victorbjorklund wrote: post #123

tleb_ wrote: post #93

Earlier quoted context omitted.

I see two differences: (1) the software stack on the server side and (2) I guess there is JS to be sent to the client side for HTMX support(?). Both those things make a difference.

The size of HTMX compressed is 10kb and very rarely changes which means it can stay in your cache for a very long time.

I'm embedded so I don't much about web stuff but sometimes I create dashboards to monitor services just for our team, tganks for introducing me to htmx. I do think html+css should be used for anything that is a document or static for longer than a typical view lasts. Arxiv is leaning towards HTML+css vs latex in acknowledgement that paper is no longer how "papers" are read. And on the other end, eBay works really well with no js right up until you get to an item's page, where it breaks. If ebay can work without js, almost anything that isn't monitoring and visualizing constant data (last few minutes of a bid, or telemetry from an embedded sensor) can work without js. I don't understand how amazon.com has gotten so slow and clunky for instance.

I have been using wasm and webgpu for visualization, partly to offload any burden from the embedded device to be monitored, but that could always be a third machine. Htmx says it supports websockets, is there a good way to have it eat a stream and plot data as telemetry, or is that time for a new tool?

Re: Engineering for Slow Internet

#200

Post by amelius » Fri, May 31, 2024, 1:36 PM UTC

meindnoch wrote: post #184

amelius wrote: post #174

I have a question about this. I have multiple instances of curl running in different terminals, all using a single slow internet connection. How can I give priority to one of these instances?

Create multiple virtual interfaces, apply different traffic shaping to them, and then use the --interface option of cURL.

Thanks, but I don't have root access to my machine.