How can network be faster than memory?
The memory number is measuring access time while the network number is measuring average bandwidth. The two values can't be compared even though they are presented using the same unit.
How “latency numbers everybody should know” decreased from 1990–2020
81–90 of 225 posts
Re: How “latency numbers everybody should know” decreased from 1990–2020
#82Okay since we're not going to improve the speed of light any time soon, here's my idea for speeding up CA to NL roundtrip: let's straight shot a cable through the center of the earth.
Assuming my math is right, it'd be a 10% faster trip, but I'd be all for seeing that tunnel!
Re: How “latency numbers everybody should know” decreased from 1990–2020
#83Earlier quoted context omitted.
What item #3 tells you is that any performance gains in the last decade and a half you've experienced have been driven by multi-core, not faster processors. And that means Amdahl's Law is more important than Moore's Law these days. Uh or storage and networking? Not sure why you would leave that out, since they're the bottleneck in many programs. The slowest things are the first things you should optimize
Yeah... SSDs are so much faster than spinning disk it's not even funny. I literally refuse to run a machine that boots its main OS from spinning disk anymore. The 60 bucks to throw an SSD into it is so incredibly cheap for what you get. My wife's work gave her a (fairly basic but still fine) thinkpad - except they left the main drive as a fucking 5400rpm hdd. Then acted like assclowns when we repeatedly showed them t…
The living room media/gaming machine at home is an 8 terabyte spinning rust. I didn't bother with a separate SSD boot partition.
It's currently been running for 23 days. Booting takes ~15 seconds even on spinning rust for a reasonable linux distro, so I'm not going to stress about those 15 seconds every couple of months.
total used free shared buff/cache available
Swap: 37Gi 617Mi 36Gi
5.1 gigabytes mostly just file cache. As a result, everything opens essentially instantly. For a bit better experience, I did a:
find ~/minecraft/world -type f -exec cat {} > /dev/null \;
to forcibly cache that, but that was all I did.
Re: How “latency numbers everybody should know” decreased from 1990–2020
#84Earlier quoted context omitted.
What item #3 tells you is that any performance gains in the last decade and a half you've experienced have been driven by multi-core, not faster processors. And that means Amdahl's Law is more important than Moore's Law these days. Uh or storage and networking? Not sure why you would leave that out, since they're the bottleneck in many programs. The slowest things are the first things you should optimize
Yeah... SSDs are so much faster than spinning disk it's not even funny. I literally refuse to run a machine that boots its main OS from spinning disk anymore. The 60 bucks to throw an SSD into it is so incredibly cheap for what you get. My wife's work gave her a (fairly basic but still fine) thinkpad - except they left the main drive as a fucking 5400rpm hdd. Then acted like assclowns when we repeatedly showed them t…
(your hard drive story is the story of my life, up to about 10 years ago. I have eliminated all but one hard drive from my house and that one doesn't spin most of the time)
Lately my vendor discussions have centered around how much work you can get done with a machine that has half a terabyte of RAM, 96 cores, and 8 NVME SSDs (it's a lot). my college box: 40MB disk, 4MB RAM, 1 66MHz CPU.
Re: How “latency numbers everybody should know” decreased from 1990–2020
#85How are people practically taking advantage of the increase in speed of SSDs these days compared to network latencies? It looks like disk caches directly at the edge with hot data would be the fastest way of doing things. I'm more familiar with the 2001-2006 era where redis-like RAM caches for really hot data made a lot of sense, but with spinning rust a disk drives, it made more sense to go over the network to a mic…
I don’t know, I have observed in my workloads, booting, game load, and building programs, that super fast ssds make almost no difference compared to cheap slow ssds. But any ssd is miraculous compared to a spinny drive Presumably video editing or something might get more of a win but I don’t know.
When I got my first NVMe SSD, I was disappointed that it wasn’t significantly faster than my SATA SSD.
But soon I realized that it was Samsung’s Magician software that made the SATA SSD competitive with an NVMe SSD via RAM caching.
Re: How “latency numbers everybody should know” decreased from 1990–2020
#86Earlier quoted context omitted.
Yeah... SSDs are so much faster than spinning disk it's not even funny. I literally refuse to run a machine that boots its main OS from spinning disk anymore. The 60 bucks to throw an SSD into it is so incredibly cheap for what you get. My wife's work gave her a (fairly basic but still fine) thinkpad - except they left the main drive as a fucking 5400rpm hdd. Then acted like assclowns when we repeatedly showed them t…
The living room media/gaming machine at home is an 8 terabyte spinning rust. I didn't bother with a separate SSD boot partition. It's currently been running for 23 days. Booting takes ~15 seconds even on spinning rust for a reasonable linux distro, so I'm not going to stress about those 15 seconds every couple of months. total used free shared buff/cache available Mem: 31Gi 4.6Gi 21Gi 158Mi 5.1Gi 25Gi Swap: 37Gi 617M…
Hah, if you can fit the whole OS plus running applications easily in RAM, and you don't boot often - fine. But you're basically doing the same thing but with extra steps :P
Re: How “latency numbers everybody should know” decreased from 1990–2020
#87Earlier quoted context omitted.
The living room media/gaming machine at home is an 8 terabyte spinning rust. I didn't bother with a separate SSD boot partition. It's currently been running for 23 days. Booting takes ~15 seconds even on spinning rust for a reasonable linux distro, so I'm not going to stress about those 15 seconds every couple of months. total used free shared buff/cache available Mem: 31Gi 4.6Gi 21Gi 158Mi 5.1Gi 25Gi Swap: 37Gi 617M…
Hah, if you can fit the whole OS plus running applications easily in RAM, and you don't boot often - fine. But you're basically doing the same thing but with extra steps :P
Well, RAM is significantly faster than even SSD, and now I don't have to muck about w/ a 2nd drive :)
Not to mention the spinning rust is cheaper.
Re: How “latency numbers everybody should know” decreased from 1990–2020
#88Earlier quoted context omitted.
It's pretty remarkable that, for efficient data processing, it's super super important to care about memory layout / cache locality in intimate detail, and this will probably be true until something fundamental changes about our computing model. Yet somehow this is fairly obscure knowledge unless you're into serious game programming or a similar field.
> Yet somehow this is fairly obscure knowledge unless you're into serious game programming or a similar field. Because the impact in optimizing hardware like that can be not so important in many applications. Getting the absolute most out of your hardware is very clearly important in game programming, but web apps where scale being served is not huge (vast majority)? Not so much. And in this context developer time is…
TBH I don't think cloud gaming is a long term solution. It might be a medium term solution for people with cheap laptops but eventually the chip in cheap laptops will be able to produce photo realistic graphics and there will be no point going any further than that
Re: How “latency numbers everybody should know” decreased from 1990–2020
#89Today I learned that I don't know any of these numbers that "every" programmer should know. Where do I turn in my programmer card, Jeff Dean?
I don’t think it’s important to know the absolute numbers but rather the relative values and rough orders of magnitude. I can’t tell you how many times I’ve had to explain to developers why their network attaches storage has higher latency than their locally attached NVME SSD
That's reasonable. My comment is a reaction to the unneeded title. Knowing relative speeds of things as well as ways to characterize and debug them is generally important.
Re: How “latency numbers everybody should know” decreased from 1990–2020
#90The "commodity network" thing is kind of weird. I'd expect that to make a 10x jump when switches went from Fast Ethernet to Gigabit (mid-late 2000s?) and then nothing. I certainly don't feel like they've been smoothly increasing in speed year after year. I'm also curious about those slow 1990s SSDs.
I was curious too, the network one is a TODO:
https://github.com/colin-scott/interactive_latencies/blob/ma...
Although I'd expect more jumps than Fast Ethernet to Gigabit, it's true the consumer space is kind of stuck on Gigabit (with 2.5 GbE just becoming more common now). Datacenters have had at least 10 GbE for many years.