Noticing when an app is only hosted in us-east-1

It shows up if you need a CDN pretty clearly when you monitor uptime from around the world.

The response time for Bitbucket for example is:

100ms from us-east

300ms from us-west

400ms from eu-central

600ms from tokyo

800ms from sydney

(numbers from OnlineOrNot)

The speed of light in a fiber optic cable is slower than light in a vacuum, about 2.14e8 m/s.

If you feel latency, it's probably not the one-direction or round-trip latency, but rather the MANY round trips that are typically required for an HTTP request. DNS is probably 2 round trips (CNAME then A), and that has to cross the ocean via your resolver of choice (8.8.8.8 or whatever) to get to the authoritative server if it's not already cached (or distributed; big DNS providers will serve your zone in many regions). Then you have to set up a TCP session, which is 1.5 round trips. Then you have to set up TLS, which varies, and make an HTTP request, and wait for the response. (I counted 5 total round trips until you see the response.)

So basically if you calculate the speed of light between two points, multiply that by 2*(2+5) = 14 in the worst case to see your time to first byte. Doing something 14 times is always going to be slow.

The underlying issue here is not so much the distance, but rather that TCP, TLS, and HTTP don't care about latency at all. (I'll ignore the application layer, which probably wants to redirect you to /verify-session-cookie and then /hey-you-logged-in for some reason. And yes, TLS1.3 has 0RTT handshakes now too, eliminating some trips.)

This is the problem that HTTP/3 aims to fix; one round trip replaces the TCP handshake, TLS handshake, and HTTP request. You shoot out a packet, you get back an HTTP response. (You still have to do the DNS lookup, so we'll call this 3 round trips total.)

> DNS is probably 2 round trips (CNAME then A), and that has to cross the ocean via your resolver of choice

I hope your DNS doesn't have to do that. Most anycast DNS should have lots of PoPs (regions) and are really fast.

CDNs usually solve a lot of the static asset issues.

The main issue is the database.

I don't buy this.

Hacker News is one of the most responsive websites I know. And it is run on a single server somewhere in the USA. While I am in Europe.

    If you have users in Sydney, Australia ... 
    ... you are floored at 104ms of latency for your request

When I open AirBnB with a clean browser cache, it takes several seconds until I see something useful. Those 104ms of latency don't make a dent.

Reddit takes over 5 seconds until the cookie banner is ready for me to click it away.

Twitter takes 6 seconds to load the homepage and display the single tweet which fits on it.

    preview images take a little longer to load

Preview images of what? Images usually should be routed through a CDN which caches them directly in the user's country. It's extremely cheap and easy to set up. Nothing compared to running an application in multiple datacenters.

Browsing HN only needs one round-trip (fetch the HTML), maybe two if you don't have it cached (fetch the CSS).

Many apps need more round-trips, by loading assets sequentially. For example: fetch the HTML, then the JS, then the JS downloads its config, then the JS fetches some assets. Latency accumulates with every round-trip.