Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How “latency numbers everybody should know” decreased from 1990–2020

colin-scott.github.io

Post reply on HN
Print view View original

71–80 of 225 posts

Re: How “latency numbers everybody should know” decreased from 1990–2020

#71

Post by not2b »

thfuran wrote: post #57
pjc50 wrote: post #49

Earlier quoted context omitted.

It also tells us that the speed of light has not increased. (well, speed of signal on a PCB track is roughly 2/3 light and determined by the transmission line geometry and the dielectric constant, but you all knew that)

Which latency are you suggesting is limited by the speed of light?

It wasn't the speed of light, it was the size of atoms that was the issue here. As old-style scaling (the kind used up until about 2003) continued, leakage power was increasing rapidly because charge carriers (electrons / holes) would tunnel through gates (I'm simplifying a bit here, other bad effects were also a factor). It was no longer possible to keep increasing clock frequency while scaling down feature size. Further reduction without exploding the power requirement meant that the clock frequency had to be left the same and transistors needed to change shape.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#72

Post by bob1029 »

Latency is everything.

I believe that sometime around 2010 we peaked on the best software solution for high performance, low-latency processing of business items when working with the style of computer architecture we have today.

https://lmax-exchange.github.io/disruptor/disruptor.html

I have been building systems using this kind of technique for a few years now and I still fail to wrap my brain around just how fast you can get 1 thread to go if you are able to get out of its way. I caught myself trying to micro-optimize a data import method the other day and made myself to do it the "stupid" way first. Turns out I was definitely wasting my time. Being able to process and put to disk millions of things per second is some kind of superpower.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#73

Post by horsawlarway »

chubot wrote: post #60
ChuckMcM wrote: post #17

Not an intuitive thing but the data is fascinating. A couple of notes of people who are confused by it: 1) The 'ns' next to the box is a graph legend not a data label (normally that would be in a box labeled legend to distinguish it from graph data) 2) The weird box and rectangle thing on the top is a slider, I didn't notice that until I was looking at the code and said "what slider?" 3) The only changes from 2005 to…

What item #3 tells you is that any performance gains in the last decade and a half you've experienced have been driven by multi-core, not faster processors. And that means Amdahl's Law is more important than Moore's Law these days. Uh or storage and networking? Not sure why you would leave that out, since they're the bottleneck in many programs. The slowest things are the first things you should optimize

Yeah... SSDs are so much faster than spinning disk it's not even funny.

I literally refuse to run a machine that boots its main OS from spinning disk anymore. The 60 bucks to throw an SSD into it is so incredibly cheap for what you get.

My wife's work gave her a (fairly basic but still fine) thinkpad - except they left the main drive as a fucking 5400rpm hdd. Then acted like assclowns when we repeatedly showed them that the machine is stalling on disk IO, while the rest of the system is doing diddly squat waiting around. I finally got tired of it and we "accidentally" spilled water on it, and somehow just the hdd stopped working (I left out the part where I'd removed it from the laptop first...). Then I just had her expense a new SSD and she no longer hates her work laptop.

Long story short - Storage speeds are incredible compared to what they were when I went to school (when 10k rpm was considered exorbitant)

Re: How “latency numbers everybody should know” decreased from 1990–2020

#74

Post by jeffbee »

genewitch wrote: post #69
jeffbee wrote: post #23

I doubt that same-facility RTT has been fixed at 500µs for 30 years. In EC2 us-east-1 I see function getDCRTT() { // Assume this doesn't change much? return 500000; // ns }

I show 180-350µs between various machines on my network, all of which have some fiber between them. devices with only a switch and copper between them somehow perform worse, but this is anecdotal because i'm not running something like smokeping! Oh, additionally between VMs i'm getting 180µs, so that looks to be my lower bound, for whatever reason. my main switches are very old, so maybe that's why.

Are you measuring that with something like ICMP ping? I think the way to gauge the actual network speed is to look at the all-time minimum RTT on a long-established TCP socket. The Linux kernel maintains this stat for normal TCP connections.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#75

Post by Archelaos »

dweez wrote: post #62

Okay since we're not going to improve the speed of light any time soon, here's my idea for speeding up CA to NL roundtrip: let's straight shot a cable through the center of the earth.

From CA you will end up off the coast of Madagaskar, and from the NL somewhere near New Zealand. You do not have to go very deep inside the earth to get straight from CA to NL.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#76

Post by formerly_proven »

greggsy wrote: post #39
gameswithgo wrote: post #25

An instructive thing here is that a lot of stuff has not improved since ~2004 or so, and working around those things that have not improved (memory latency from ram all the way down to l1 cache really) requires fine control of memory layout and minimizing cache pollution, which is difficult to do with all of our popular garbage collected languages, even harder with languages that don't offer memory layout controls, a…

It’s interesting that L2 cache has basically been steady at 2MB/core since 2004 aswell. It hasn’t changed speed in that time, but is still an order of magnitude faster than memory across that whole timeframe. Does this suggest that the memory speed bottleneck means that there simply hasn’t been a need to increase availability of that faster cache?

Some of these numbers are clearly wrong. Some of the old latency numbers seem somewhat optimistic (e.g. 100 ns main memory ref in 1999), some of the newer ones are pessimistic (e.g. 100 ns main memory ref in 2020). The bandwidth for disks is clearly wrong, as it claims ~1.2 GB/s for a hard drive in 2020. The seek time is also wrong. It crossed 10 ms in 2000 and has reduced to 5 ms in 2010 and is 2 ms for 2020. Seems like linear interpolation to me. It's also unclear what the SSD data is supposed to mean before ~2008 as they were not really a commercial product before then. Also, for 2020 the SSD transfer rate is given as over 20 GB/s. Main memory bandwidth is given as 300+ GB/s.

Cache performance has increased massively. Especially bandwidth, not reflected in a latency chart. Bandwidth and latency are of course related; just transferring a cache line over a PC66 memory bus takes a lot longer than 100 ns. The same transfer on DDR5 takes a nanosecond or so, which leaves almost all of the latency budget for existential latency.

edit: https://github.com/colin-scott/interactive_latencies/blob/ma...

The data on this page is simply extrapolated using formulas and guesses.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#77

Post by not2b »

rvr_ wrote: post #50

20 years without meaningful improvements on memory access ?

It takes at least one clock cycle to do anything, and clock frequency stopped increasing in the 2003-2005 time frame, mainly because of the horrible effects on power with very small feature size.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#78

Post by efuquen »

TillE wrote: post #56
gameswithgo wrote: post #25

An instructive thing here is that a lot of stuff has not improved since ~2004 or so, and working around those things that have not improved (memory latency from ram all the way down to l1 cache really) requires fine control of memory layout and minimizing cache pollution, which is difficult to do with all of our popular garbage collected languages, even harder with languages that don't offer memory layout controls, a…

It's pretty remarkable that, for efficient data processing, it's super super important to care about memory layout / cache locality in intimate detail, and this will probably be true until something fundamental changes about our computing model. Yet somehow this is fairly obscure knowledge unless you're into serious game programming or a similar field.

> Yet somehow this is fairly obscure knowledge unless you're into serious game programming or a similar field.

Because the impact in optimizing hardware like that can be not so important in many applications. Getting the absolute most out of your hardware is very clearly important in game programming, but web apps where scale being served is not huge (vast majority)? Not so much. And in this context developer time is more valuable when you can throw hardware at the problem for less.

Traditional game programming you had to run on the hardware people used to play, you are constrained by the client's abilities. Cloud gaming might(?) be changing some of that, but GPUs are super expensive too compared to the rest of the computing hardware. Even in that case the amounts of data you are pushing you need to be efficient within the context of the GPU, my feeling is it's not easily horizontally scaled.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#79

Post by throwawaylinux »

greggsy wrote: post #39
gameswithgo wrote: post #25

An instructive thing here is that a lot of stuff has not improved since ~2004 or so, and working around those things that have not improved (memory latency from ram all the way down to l1 cache really) requires fine control of memory layout and minimizing cache pollution, which is difficult to do with all of our popular garbage collected languages, even harder with languages that don't offer memory layout controls, a…

It’s interesting that L2 cache has basically been steady at 2MB/core since 2004 aswell. It hasn’t changed speed in that time, but is still an order of magnitude faster than memory across that whole timeframe. Does this suggest that the memory speed bottleneck means that there simply hasn’t been a need to increase availability of that faster cache?

Bigger caches could help but as a rule of thumb cache hit rate increases approximately with the square root of cache size, so it diminishes. Then the bigger you make a cache, the slower it tends to be so at some point you could make your system slower by making your cache bigger and slower.
Post reply on HN