Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
motherboard.vice.com
Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
1–10 of 24 posts
Re: Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
#2I try to refrain from purely negative commentary on articles, but yuck! How can one hope to say anything useful about "memory performance" without once using the terms "latency" or "bandwidth"? Memory performance is getting higher in the same way that processor performance is going up:
http://www.corsair.com/en-us/blog/2015/september/ddr3_vs_ddr...
Latency is to frequency as bandwidth is to parallelism. Single core CPU frequencies are relatively stable, but parallelism offers the opportunity to get more done each cycle. Latency for random access is holding quite steady, but caches are getting bigger and faster, and bandwidth is going up.
The key is figuring out how to write software that takes advantage of spatial locality and available bandwidth rather than getting choked by the latency. This is hard in the same way that taking advantage of multiple cores is hard: it requires a different approach, but is not a fundamental limitation.
Lots of memory access isn't the problem. Long latency isn't the problem. The problem is designing your program so that it generates lots of long latency memory accesses and then grinds to a halt in the presence of this latency. I think the summary from this recent paper is spot on:
Our conclusion is contrary to our expectations and to
previous findings and goes against conventional wisdom,
which states that accesses to RAM are slow, and should be
minimized. A more accurate statement, that accounts for our
findings, is that accesses to RAM have high latency and
this latency needs to be mitigated.
Re: Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
#3A much bigger problem "holding up Moore's Law Progression" is the failure of Dennard scaling and the fact that voltage scaling is hitting the threshold voltage limit (where sub-threshold leakage current increases significantly) as we move to smaller technology nodes. This means we can build bigger chips but we don't necessarily have the power budget to power up all parts of it at the same time (these could be cores, pipeline structures, etc). The architecture community has written a lot on this "Dark Silicon" problem if anyone wants to read further.
Re: Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
#4Moore's Law is about transistor count. Not about frequency, bandwidth or "speed"... Is everybody ignoring this fact?
Re: Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
#5I try to refrain from purely negative commentary on articles, but yuck! How can one hope to say anything useful about "memory performance" without once using the terms "latency" or "bandwidth"? Memory performance is getting higher in the same way that processor performance is going up: http://www.corsair.com/en-us/blog/2015/september/ddr3_vs_ddr... Latency is to frequency as bandwidth is to parallelism. Single core C…
> I try to refrain from purely negative commentary on articles, but yuck! How can one hope to say anything useful about "memory performance" without once using the terms "latency" or "bandwidth"?
Yeah, it is pretty clear the author didn't really understand much of the topic. Even if you have lightning fast memory, there's still the issue of connecting it. Long traces and multiple levels of multiplexing are going to add latency no matter how amazing memory tech is being employed. Before long, to increase bandwidth, memory needs to be integrated in the same package as the CPU if not on the same die. At least large eDRAM-style 'caches'. You can't economically run 1024 traces just for memory on the mainboard PCB!
> The key is figuring out how to write software that takes advantage of spatial locality and available bandwidth rather than getting choked by the latency. This is hard in the same way that taking advantage of multiple cores is hard: it requires a different approach, but is not a fundamental limitation.
Indeed, of course this requires skilled labor for now, until compilers catch up one day. And unfortunately code that truly requires random access is just not going to perform well on modern hardware. Up until early nineties, memory was faster than processors. Random access was just fine. Not so anymore.
It's also easy to get bandwidth limited with SSE and AVX. Although line fill etc. buffers seem to often bottleneck first per CPU core. For scalar code, being bandwidth limited is not going to happen.
The issue nowadays is that machines are so unique snowflakes performance and configuration wise. It's not hard to max out a single configuration, but it is hard to write something that performs decently across different system configurations. There are like 30 instruction set extensions for x86, variations in reorder buffer depth, variations in cache latency, size and associativity. And of course memory bus configurations, number and interleaving of memory channels, NUMA, DRAM memory page size (1, 2, 4 kB), etc.
For example unlike 8-way associative L2 cache on Sandy Bridge, Haswell, etc., on Skylake it is now 4-way instead. Code that is optimised for 8-way L2 cache might be pathologically invalidating L2 cache lines on Skylake.
Those aspects matter, because high performance is often a balancing act between available features, bandwidth and CPU power.
Re: Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
#6I try to refrain from purely negative commentary on articles, but yuck! How can one hope to say anything useful about "memory performance" without once using the terms "latency" or "bandwidth"? Memory performance is getting higher in the same way that processor performance is going up: http://www.corsair.com/en-us/blog/2015/september/ddr3_vs_ddr... Latency is to frequency as bandwidth is to parallelism. Single core C…
> I try to refrain from purely negative commentary on articles, but yuck! How can one hope to say anything useful about "memory performance" without once using the terms "latency" or "bandwidth"? Yeah, it is pretty clear the author didn't really understand much of the topic. Even if you have lightning fast memory, there's still the issue of connecting it. Long traces and multiple levels of multiplexing are going to a…
And while you're at it, it's hard to figure out what's going on in your system. I've spent some time looking, and have not yet found anything resembling a bandwidth monitor for main memory.
Re: Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
#7No it's not.
Since skylake comes in both DDR4 and DDR3 motherboard versions, various benchmarks have come out to test the difference for the state-of-the-art 14nm cpu with the "improved" memory vs the "old" memory.
And the difference is often only 1-2%
Unless maybe the goal should be to put 32gb of memory right on the cpu die
Re: Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
#8Earlier quoted context omitted.
> I try to refrain from purely negative commentary on articles, but yuck! How can one hope to say anything useful about "memory performance" without once using the terms "latency" or "bandwidth"? Yeah, it is pretty clear the author didn't really understand much of the topic. Even if you have lightning fast memory, there's still the issue of connecting it. Long traces and multiple levels of multiplexing are going to a…
And while you're at it, it's hard to figure out what's going on in your system. I've spent some time looking, and have not yet found anything resembling a bandwidth monitor for main memory.
It's possible, but depending on your OS and processor it might be quite difficult. There are "Uncore" performance counters for this, but the means of access has been changing from generation to generation and the software sometimes lags behind.
For Linux and Haswell EP, I've had luck with likwid: https://github.com/RRZE-HPC/likwid/wiki/Haswell-EP#memory-co...
You also might have luck with Andi Kleen's pmu-tools: https://github.com/andikleen/pmu-tools/blob/master/ucevent/R...
I don't know for sure if Intel's VTune supports these counters, but I'd presume it would: https://software.intel.com/en-us/intel-vtune-amplifier-xe
Re: Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
#9No it's not. Since skylake comes in both DDR4 and DDR3 motherboard versions, various benchmarks have come out to test the difference for the state-of-the-art 14nm cpu with the "improved" memory vs the "old" memory. And the difference is often only 1-2% Unless maybe the goal should be to put 32gb of memory right on the cpu die
Different software needs different resources.
Unsurprisingly, if the software in question is not bandwidth limited, providing more bandwidth is not going to speed it up. Most software is like that.
It also truly depends on how it was optimized. The software being tested was likely optimized for previous gen configurations. It might very well favor a bit more computation over higher memory bandwidth usage.
Wait until developers optimize against DDR4 Skylake systems, you might start to see 5-10% difference at that point. Truly bandwidth limited code can run up to about 40% faster on a DDR4 system, assuming typical 1600 MHz DDR3 and 2400 MHz DDR4.
Re: Memory Is Holding Up the Moore's Law Progression of Processing Power (2014)
#10Moore's Law is about transistor count. Not about frequency, bandwidth or "speed"... Is everybody ignoring this fact?
In the past, the increase of transistor count and density meant that a CPU could be designed to run at a higher frequency and have cleverer circuitry which allowed it to do more each cycle. This is the source of the misconception that Moore's Law is directly tied to performance.
Looking at a modern Intel chip, the CPU cores take about 30% of the area, with the rest of the transistors spent elsewhere (L3 cache, integrated graphics, system and memory controllers).