Live data from Hacker News

How “latency numbers everybody should know” decreased from 1990–2020

colin-scott.github.io

151–160 of 225 posts

Re: How “latency numbers everybody should know” decreased from 1990–2020

#151
post #62

Okay since we're not going to improve the speed of light any time soon, here's my idea for speeding up CA to NL roundtrip: let's straight shot a cable through the center of the earth.

the internet doesn't travel at the speed of light. i think it's around 2/3 the speed of light. could still reduce that ping by about 33%

Re: How “latency numbers everybody should know” decreased from 1990–2020

#152
post #27

Amazing performance improvements, except no improvement at all on the packet roundtrip time to Netherlands. Someone should really work on that.

People have tried: https://www.cs.cmu.edu/~xia/resources/Documents/Singla_hotne...

Re: How “latency numbers everybody should know” decreased from 1990–2020

#153

How are people practically taking advantage of the increase in speed of SSDs these days compared to network latencies? It looks like disk caches directly at the edge with hot data would be the fastest way of doing things. I'm more familiar with the 2001-2006 era where redis-like RAM caches for really hot data made a lot of sense, but with spinning rust a disk drives, it made more sense to go over the network to a mic…

I don’t know, I have observed in my workloads, booting, game load, and building programs, that super fast ssds make almost no difference compared to cheap slow ssds. But any ssd is miraculous compared to a spinny drive Presumably video editing or something might get more of a win but I don’t know.

I'm talking about back in the day when I worked at Amazon with thousands of servers and in the 2001-2006 era we had basically no SSDs and it was all spinny rust (set the slider to 2006 and that is about my mental model). Spinny rust was always shit, so it was all about RAM and network and then the long tail siting on disk.

I'm wondering how datacenter SSDs impact architectural decisions with e.g. 'microservices' by having that new latency layer which is cheaper than RAM and faster than network.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#155

Not an intuitive thing but the data is fascinating. A couple of notes of people who are confused by it: 1) The 'ns' next to the box is a graph legend not a data label (normally that would be in a box labeled legend to distinguish it from graph data) 2) The weird box and rectangle thing on the top is a slider, I didn't notice that until I was looking at the code and said "what slider?" 3) The only changes from 2005 to…

Actually, this data is all false from 2012 onwards. The code was written back in 2012 and is using numbers from back then to _guess_ what future numbers would look like! (I was fooled by it too) See https://github.com/colin-scott/interactive_latencies/issues/...

Even the 2012 data is suspect.

"Seek + rotational delay halves every 10 years" just isn't true. We got rid of the >7200 drives and are basically sitting still on that metric. And hard drives weren't at 400MB/s in 2012 either. (The disk bandwidth section also cites a 2012 presentation with correct numbers, but something went wrong in translation.)

Re: How “latency numbers everybody should know” decreased from 1990–2020

#156
post #27

Amazing performance improvements, except no improvement at all on the packet roundtrip time to Netherlands. Someone should really work on that.

The great circle distance between SF and Amsterdam is about 5,450 miles. A round trip is 10,900 miles. At the speed of light, that's 58.5 milliseconds. So in theory, infinitely efficient computers (and some sort of light-in-a-vacuum wire) and networking equipment could knock 60% off that time, but no more. However, I suspect one big problem is that there are a number of hops between the Netherlands and San Francisco.…

You can get it down another 10% again, or so if you dig straight through.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#157
post #114

Earlier quoted context omitted.

The interesting part is that it say nothing about performance . Single-core benchmarks have gotten significantly faster over that time period. If anything, the takeaway is that things like memory/cache access, branch prediction failures, and mutexes have gotten more expensive. They didn't scale while the rest of the CPU sped up! But even that isn't really true, because it doesn't tell you anything about branch predic…

This is a valid point, and this is an overly long response because it distracts me from watching frightening current events. There are two ways to look at these sorts of numbers, "CPU performance" and "Systems performance". To give an example from my history; NetApp was dealing with the Pentium P4 being slower than the Pentium 3 and looking at how that could be. All of the performance numbers said it should be faster…

Just for anyone curious, the way computer engineers / processor architects deal with the above is to measure the effective Instructions Per Cycle on a battery of benchmarks, captured traces/samples, synthetic workloads etc.

The P4 was a design that reached higher cycle rates, but at the cost of a lower IPC such that the net performance was a regression. In particular the pipeline depth was such that any dependency stalls or flushes eroded CPI severely.

There's a lot that went into this mistake. Having a high Ghz number for marketing purposes was part of it, but also in this era Intel Labs had demonstrators of 10 Ghz ALUs running, so there was some reason to think this architecture might prevail eventually. In any case, by the time it launched retail, Intel knew they had a lemon, but couldn't change course without building up a new microarchitecture.

Back in the early 2000's several of the P4 architects were quite active on comp.arch, and quite forthright about the mistakes they made. There were lots of interesting and insightful posts, but sadly I think you'd have to do some considerable digging in the deja archives or whatever it's called now to find the subthreads.

If you're interested in this sort of stuff, Hennessy and Patterson's Computer Architecture: A Quantitative Approach is the go to textbook.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#158

Not an intuitive thing but the data is fascinating. A couple of notes of people who are confused by it: 1) The 'ns' next to the box is a graph legend not a data label (normally that would be in a box labeled legend to distinguish it from graph data) 2) The weird box and rectangle thing on the top is a slider, I didn't notice that until I was looking at the code and said "what slider?" 3) The only changes from 2005 to…

At what point did Joy's Law -- "2^(year-1884) MIPS" -- break with reality? https://medium.com/@donhopkins/bill-joys-law-2-year-1984-mil... https://en.wikipedia.org/wiki/Joy%27s_law_(computing)

> At what point did Joy's Law -- "2^(year-1884) MIPS" -- break with reality?

It was never really true. PS It is formulated as 1984 not 1884, though. 1884 throws off MIPS calculations by 2^100 or 10^30.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#159
post #78

Earlier quoted context omitted.

> Yet somehow this is fairly obscure knowledge unless you're into serious game programming or a similar field. Because the impact in optimizing hardware like that can be not so important in many applications. Getting the absolute most out of your hardware is very clearly important in game programming, but web apps where scale being served is not huge (vast majority)? Not so much. And in this context developer time is…

IMO we are only scratching the surface of cloud gaming so far. Right now it’s pretty much exclusively lift-and-shift, hosted versions of the same game, in many cases running on consumer GPUs. Cloud gaming allows for the development of cloud-native games that are so resource intensive (potentially architected so that more game state is shared across users) that they would not possible to implement on consumer hardware…

I've thought about this off and on, and there's certainly interesting possibilities. You can imagine a cloud renderer that does something like a global scatter / photon mapping pass, while each client's session on the front end tier does an independent gather/render. Obviously there's huge problems to making something like this work practically, but just mention it as an example of the sort of more novel directions we should at least consider.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#160

Earlier quoted context omitted.

Photo realistic graphics ought to be enough for anybody? This seems unlikely, there's so many aspects to graphical immersion that there's still plenty of room for improvement and AAA games will find them. Photo realistic graphics is a rather vague target, it depends on what and how much you're rendering. Then you need to consider that demand grows with supply, with eg. stuff like higher resolutions, even higher refre…

There are diminishing returns. If a laptop could play games at the quality of a top end PC today, would people really want to pay for an external streaming service, deal with latency, etc just so they can get the last 1% of graphical improvements? We have seen there are so many aspects of computing where once it’s good enough, it’s good enough. Like how onboard DACs got good enough that even the cheap ones are suffic…

> If a laptop could play games at the quality of a top end PC today, would people really want to pay for an external streaming service, deal with latency, etc just so they can get the last 1% of graphical improvements?

Think of it a different direction: if/when cloud rendering AAA graphics is practical, you can get a very low friction Netflix like experience where you just sit down and go.

Post reply on HN