Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How “latency numbers everybody should know” decreased from 1990–2020

colin-scott.github.io

Post reply on HN
Print view View original

91–100 of 225 posts

Re: How “latency numbers everybody should know” decreased from 1990–2020

#91

Post by dekhn »

formerly_proven wrote: post #76
greggsy wrote: post #39

Earlier quoted context omitted.

It’s interesting that L2 cache has basically been steady at 2MB/core since 2004 aswell. It hasn’t changed speed in that time, but is still an order of magnitude faster than memory across that whole timeframe. Does this suggest that the memory speed bottleneck means that there simply hasn’t been a need to increase availability of that faster cache?

Some of these numbers are clearly wrong. Some of the old latency numbers seem somewhat optimistic (e.g. 100 ns main memory ref in 1999), some of the newer ones are pessimistic (e.g. 100 ns main memory ref in 2020). The bandwidth for disks is clearly wrong, as it claims ~1.2 GB/s for a hard drive in 2020. The seek time is also wrong. It crossed 10 ms in 2000 and has reduced to 5 ms in 2010 and is 2 ms for 2020. Seems…

The oldest latency numbers were based on actual hardware Google had on hand at the time. Most of them came from microbenchmarks checked into google3. Occasionally an engineer would announce they had found an old parameter tuned for pentium and cranking it up got another 1-2% performance gain on important metrics.

Many of the newer numbers could be based on existing google hardware; for example, Google deployed SSDs in 2008 (custom designed and manufactured even before then) because hard drive latency wasn't getting any better. That opened up a bunch of new opportunities. I worked on a project that wanted to store a bunch of data, Jeff literally came to me with a code that he said "compressed the data enough to justify storing it all on flash, which should help query latency" (this led to a patent!).

Re: How “latency numbers everybody should know” decreased from 1990–2020

#92

Post by genewitch »

jeffbee wrote: post #74
genewitch wrote: post #69

Earlier quoted context omitted.

I show 180-350µs between various machines on my network, all of which have some fiber between them. devices with only a switch and copper between them somehow perform worse, but this is anecdotal because i'm not running something like smokeping! Oh, additionally between VMs i'm getting 180µs, so that looks to be my lower bound, for whatever reason. my main switches are very old, so maybe that's why.

Are you measuring that with something like ICMP ping? I think the way to gauge the actual network speed is to look at the all-time minimum RTT on a long-established TCP socket. The Linux kernel maintains this stat for normal TCP connections.

Yes, ICMP ping. I don't have any long established sockets anyhow so i don't know how useful my set of those stats would be.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#93

Post by teraflop »

genewitch wrote: post #65
aeyes wrote: post #58

Earlier quoted context omitted.

Could LEO satellite networks like Starlink with inter-satellite links reduce the roundtrip time?

the radius of the arc in low earth orbit (or whatever) is going to be larger than the arc across the atlantic ocean. As no one has ever said: "I'll take glass over gas, thanks."

Yeah, but the faster speed of light through vacuum can still be enough to make up for the longer path.

Back-of-the-envelope calculation: at an altitude of 550km and minimum elevation of 25 degrees above the horizon, a single Starlink hop could cover a maximum of 1880km of distance along the earth's surface, with a ground-satellite-ground length of 2068km.

So the total distance traveled through space is about 10% farther, but the signal goes 50% faster than through fiber, which is enough to cut your round-trip time from about 19ms to 14ms (plus any extra latency introduced by routers). That's nothing to sneeze at.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#95

Post by jmyeet »

This could be a really great visualization and really interesting but I literally nad to look at he comments here to figure out how to read it and use it. For example, you can edit the year text box.

I would suggest adding graphs to show how different things have changed over time.

it is interesting that the only major improvement over the last 15+ years seems to have been the invention of SSDs.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#96

Post by nightpool »

thfuran wrote: post #57
pjc50 wrote: post #49

Earlier quoted context omitted.

It also tells us that the speed of light has not increased. (well, speed of signal on a PCB track is roughly 2/3 light and determined by the transmission line geometry and the dielectric constant, but you all knew that)

Which latency are you suggesting is limited by the speed of light?

"Packet roundtrip CA to Netherlands"

Re: How “latency numbers everybody should know” decreased from 1990–2020

#97

Post by dekhn »

bmitc wrote: post #13

Today I learned that I don't know any of these numbers that "every" programmer should know. Where do I turn in my programmer card, Jeff Dean?

This was a doc handed to incoming Google engineers (along with Jeff's resume) to help assimilate them. MOst if not all of these came up at some point in the development of google.com and making decisions about (for example) whether to send an intercontinental RPC or look something up locally, or to put some part of the web index in RAM/disk.

Don't read it as "you should have known this", read it as "you should know this" to be a more effective programmer. These sorts of guidelines have been invaluable for making arch decisions throughout my career.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#98

Post by faeyanpiraat »

dekhn wrote: post #84
horsawlarway wrote: post #73

Earlier quoted context omitted.

Yeah... SSDs are so much faster than spinning disk it's not even funny. I literally refuse to run a machine that boots its main OS from spinning disk anymore. The 60 bucks to throw an SSD into it is so incredibly cheap for what you get. My wife's work gave her a (fairly basic but still fine) thinkpad - except they left the main drive as a fucking 5400rpm hdd. Then acted like assclowns when we repeatedly showed them t…

(your hard drive story is the story of my life, up to about 10 years ago. I have eliminated all but one hard drive from my house and that one doesn't spin most of the time) Lately my vendor discussions have centered around how much work you can get done with a machine that has half a terabyte of RAM, 96 cores, and 8 NVME SSDs (it's a lot). my college box: 40MB disk, 4MB RAM, 1 66MHz CPU.

I assume you meant half a terabyte of ram

Re: How “latency numbers everybody should know” decreased from 1990–2020

#99

Post by mirker »

horsawlarway wrote: post #73
chubot wrote: post #60

Earlier quoted context omitted.

What item #3 tells you is that any performance gains in the last decade and a half you've experienced have been driven by multi-core, not faster processors. And that means Amdahl's Law is more important than Moore's Law these days. Uh or storage and networking? Not sure why you would leave that out, since they're the bottleneck in many programs. The slowest things are the first things you should optimize

Yeah... SSDs are so much faster than spinning disk it's not even funny. I literally refuse to run a machine that boots its main OS from spinning disk anymore. The 60 bucks to throw an SSD into it is so incredibly cheap for what you get. My wife's work gave her a (fairly basic but still fine) thinkpad - except they left the main drive as a fucking 5400rpm hdd. Then acted like assclowns when we repeatedly showed them t…

That’s fine until you have to store the whole world’s cat videos on Youtube at minimal cost.

Re: How “latency numbers everybody should know” decreased from 1990–2020

#100

Post by horsawlarway »

capitainenemo wrote: post #87
horsawlarway wrote: post #86

Earlier quoted context omitted.

Hah, if you can fit the whole OS plus running applications easily in RAM, and you don't boot often - fine. But you're basically doing the same thing but with extra steps :P

Well, RAM is significantly faster than even SSD, and now I don't have to muck about w/ a 2nd drive :) Not to mention the spinning rust is cheaper.

Sure, but that RAM isn't :)

I can get a 60gb ssd for under 20 bucks (I just checked 18.99 on amazon - even I'm a little surprised at how cheap that is)

That 32gb of ram is at least a crisp hundred (which also - fucking mind blowing. First time I ever added a part to a PC, it was to add 256mb of ram, and that was huge deal)

Post reply on HN