How “latency numbers everybody should know” decreased from 1990

Re: How “latency numbers everybody should know” decreased from 1990–2020

#21

Post by ianai » Thu, Mar 03, 2022, 9:38 PM UTC

ChuckMcM wrote: post #17

Not an intuitive thing but the data is fascinating. A couple of notes of people who are confused by it: 1) The 'ns' next to the box is a graph legend not a data label (normally that would be in a box labeled legend to distinguish it from graph data) 2) The weird box and rectangle thing on the top is a slider, I didn't notice that until I was looking at the code and said "what slider?" 3) The only changes from 2005 to…

"And that means Amdahl's Law is more important than Moore's Law these days."

idk, sure seems like we could have 1-2 cores (permanently pegged?) at 5 ghz for UI/UX then ($money / $costPerCores) number of cores for showing off/"performance" by now. But the OEMs haven't gone that way.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#22

Post by nhoughto » Thu, Mar 03, 2022, 9:39 PM UTC

the-dude wrote: post #5

The slider is very bad UX : I missed it too at first. It is not pronounced enough, partly because it is all the way to the right. A former boss would say : make it red .

Oh right

I was trying to see where the comparison was, totally missed the slider, thanks!

Re: How “latency numbers everybody should know” decreased from 1990–2020

#23

Post by jeffbee » Thu, Mar 03, 2022, 9:39 PM UTC

I doubt that same-facility RTT has been fixed at 500µs for 30 years. In EC2 us-east-1 I see

    function getDCRTT() {
        // Assume this doesn't change much?
        return 500000; // ns
    }

Re: How “latency numbers everybody should know” decreased from 1990–2020

#24

Post by greggsy » Thu, Mar 03, 2022, 9:40 PM UTC

the-dude wrote: post #5

The slider is very bad UX : I missed it too at first. It is not pronounced enough, partly because it is all the way to the right. A former boss would say : make it red .

I sympathise that the site probably wasn’t designed with mobile in mind, but it’s impossible to go beyond 2015 without hitting the GitHub link.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#25

Post by gameswithgo » Thu, Mar 03, 2022, 9:40 PM UTC

An instructive thing here is that a lot of stuff has not improved since ~2004 or so, and working around those things that have not improved (memory latency from ram all the way down to l1 cache really) requires fine control of memory layout and minimizing cache pollution, which is difficult to do with all of our popular garbage collected languages, even harder with languages that don't offer memory layout controls, and jits and interpreters add further difficulty.

To get the most out of modern hardware you need to:

* minimize memory usage/hopping to fully leverage the CPU caches

* control data layout in memory to leverage the good throughput you can get when you access data sequentially

* be able to fully utilize multiple cores without too much overhead and with minimal risk of error

For programs to run faster on new hardware, you need to be able to do at least some of those things.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#26

Post by CSSer » Thu, Mar 03, 2022, 9:40 PM UTC

Is a commodity network a local network?

Re: How “latency numbers everybody should know” decreased from 1990–2020

#27

Post by kens » Thu, Mar 03, 2022, 9:41 PM UTC

Amazing performance improvements, except no improvement at all on the packet roundtrip time to Netherlands. Someone should really work on that.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#28

Post by gameswithgo » Thu, Mar 03, 2022, 9:42 PM UTC

ChuckMcM wrote: post #17

Not an intuitive thing but the data is fascinating. A couple of notes of people who are confused by it: 1) The 'ns' next to the box is a graph legend not a data label (normally that would be in a box labeled legend to distinguish it from graph data) 2) The weird box and rectangle thing on the top is a slider, I didn't notice that until I was looking at the code and said "what slider?" 3) The only changes from 2005 to…

There is some improvement from processors being faster, as more instructions are done at once and more instructions get down towards that 1ns latency that l1 caches provide. You see it happen in real life but the gains are small.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#29

Post by warmwaffles » Thu, Mar 03, 2022, 9:43 PM UTC

Now we just need to either beat the speed of light, or speed light up. (thanks futurama)

Re: How “latency numbers everybody should know” decreased from 1990–2020

#30

Post by morelisp » Thu, Mar 03, 2022, 9:44 PM UTC

zachberger wrote: post #18

bmitc wrote: post #13

Today I learned that I don't know any of these numbers that "every" programmer should know. Where do I turn in my programmer card, Jeff Dean?

I don’t think it’s important to know the absolute numbers but rather the relative values and rough orders of magnitude. I can’t tell you how many times I’ve had to explain to developers why their network attaches storage has higher latency than their locally attached NVME SSD

The absolute numbers are also important. I can't tell you how many times I've had someone coming from a front-end world tells me 5ms for some trivial task (e.g. sorting a 1000ish element list) is "fast" just because it happened faster than their reaction time.