Live data from Hacker News

What Every Programmer Should Know About Memory (2007) [pdf]

akkadia.org

61–70 of 101 posts

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#61

Earlier quoted context omitted.

>"Although x86 specific, I'd also add x86-64 has 48-bit physical pointers: the top 16-bits are basically ignored by the current virtual memory system." Could you elaborate on this? Which pointers exactly?

All pointers in user-space are 48-bits. All 48-bit pointers are translated by the page-directory virtual memory system into a real physical location. There's an extension to use 55-bits or 56-bits... I forget exactly. But I don't think its actually been implemented yet on any CPU yet. EDIT: It was Intel's 57-bit memory: https://en.wikipedia.org/wiki/Intel_5-level_paging . Yeah, I knew it was a weird number. But I gue…

Oh I see what you are saying but that's not really just a pointer or a userspace thing, that 48 bit limit is simply imposed by the x86-64 CPU vendors in no?

With 48 bits you can still address 256TB of memory. I guessing that from a practical and financial point of view it probably made little sense for vendors to build a CPU that enabled addressing the full 64 bits. At least for now.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#62

Earlier quoted context omitted.

Can you elaborate why the second is faster in the general case? Is this an issue of row major vs column major orientation? Also does your example also hold in the absence of SIMD? Thanks.

There's at least 2 reasons (+1 non-reason) why it is slow. 1. L1 cache lines are 64-bytes long. By fetching column-wise, you are wasting the bandwidth between L1 and main-memory. L1 cache will always fetch 64-bytes. By moving "with" the cache, you allow the L1 --> Main Memory data-transfers to be far more efficient. 2. Virtual Memory is translated by the TLB before it returns the actual value. Moving within a 4kB pag…

Make perfect sense, thanks for the concise explanations. Cheers.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#63
post #60

Earlier quoted context omitted.

> I'm not saying it's not interesting, but it's not going to change how I write my for-loops and if statements or make calls to BigTable, etc. You might be surprised at how much changing your for loops can be when accessing matrix data. for(int i=0; i Test that out, and you'll see a major performance improvement. ----------- If someone is writing high performance code, the #1 goal is to be able to read your profiler'…

I have to admit, I was skeptical but my complier did pick SIMD for the second and not the first and it did make a huge difference. % cat t.c #include #include #include double get_time() { struct timeval t; struct timezone tzp; gettimeofday(&t, &tzp); return t.tv_sec + t.tv_usec*1e-6; } int blah[SIZE][SIZE]; int whatever() { static int i=0; return ++i; } int main() { double t0 = get_time(); for(int i=0; i 2.8G Core i7…

n.b. there would be a difference even without vectorization

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#64
post #4

Earlier quoted context omitted.

If you are objecting to "every," then sure, not every programmer needs to know anything about memory. You can program without knowing that there is such a thing. But it's a fun title, and an excellent resource.

No, he's right. It's more than simply semantics. A better title would be "an extremely overwhelming of programmers don't need to know THIS much about memory"

Or perhaps some more positive phrasings:

- If you know this much about memory, you'll know more than 99% of programmers.

- What 99% of programmers don't know about memory.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#65
post #14

There are some salty comments here, but I think the context is important. This paper passed across my desk in early 2008 when I was doing HFT stuff. It might be a bit of a stretch to say that the reason people are taught about cache lines in most CS programs is because of this paper, but at the time this paper was written, this was really specialized knowledge and groundbreaking to most software developers. This woul…

He mentions his reason for the title:

> The title of this paper is an homage to David Goldberg’s classic paper “What Every Computer Scientist Should Know About Floating-Point Arithmetic”

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#66

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

I would argue that the overclocker needs to know more about these details than most programmers, yes. (Overclockers actually tweak these values to maximize the performance of their computer). But any high-performance programmer needs to understand the RAS / CAS / PRE cycle, if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient. If you are accessing RAM randomly, you b…

I am all for knowing things just to know things...but why does a programmer need to know WHY the streaming of data is efficient?

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#67
post #32

Earlier quoted context omitted.

Again, you say these are "must haves." But in the real world of software, none of these would make any difference to most of the software being built today, much of which is being build in higher level languages for web applications/web sites/microservices etc. Let's say someone is building a micro-service in C#.Net. Why would any of this stuff matter to them? The company cares about features and moving forward quick…

Sure, you don't really need to know any of this stuff unless you're actually needing to optimize code beyond the lowest hanging fruit. My list presupposes you've hit a wall and need the best performance you can get. That's not always the case, but I certainly wouldn't say that these optimizations don't make a difference in the "real world of softare." False sharing alone can be the difference between a 16x paralleliz…

Right, the question then becomes "What percentage of programmers are going to need to optimize code beyond the lowest hanging fruit"

I don't know exactly what that number is going to be, but I do know it is going to be a lot less than 100%.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#68

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

Every programmers should know something about memory, because software are becoming bloat [1] [2] because of a new generation of programmers who doesn't optimize their code. So, of course, you had a wonderful career, but it doesn't prove that you write efficient code [1] http://www.rntz.net/post/against-software-development.html [2] http://tonsky.me/blog/disenchantment/

I think the argument is that most programmers don't need to write efficient code.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#69

Earlier quoted context omitted.

Computer Science is a different discipline to Software Engineering. I would argue that sure, if you want to be a PhD, academic, researcher etc, in Computer Science, then reading said article might be useful. For engineers working at companies whose goal is to get product to market faster, coding ruby on rails, javascript, making web apps, websites, it's far more useful knowing your web frameworks well and being able…

The referenced work talks about floating point errors that both computer scientists and field engineers need to avoid. Many folks in both camps screw up on floating point code without such guidance. So, at least that part is potentially helpful. You just proved their point about how you'd reply, though.

Proved their point? Their point didn't make much sense.

What actual engineers NEED to know about floating point numbers could be put on a single A4 page. It doesn't have to be 50 pages, where many of those pages are full of equations that 99% of engineers don't understand.

My point is, because you seem to have trouble grasping this. Most real-world programmers, working at startups or bigger companies, making their WPF applications or Spring applications, or react/jquery/angular apps, or Swift IOS apps, etc, do not NEED to know 99% of this document. For most real-life cases all they need to know is "Use the decimal type if you're dealing with money" or something along those lines.

Why do I know I'm correct? Because I work in an organization with 800-1000 engineers, and I know a fair few of them myself, and I bet maybe 1 person in the org has read this doc fully (and even this is a stretch). But the company stills makes billions in revenue every year like clockwork and the world keeps on spinning.

Hence, this document is for theorists and academics, not for the average engineer making enterprise business applications. If it is for an engineer, it's for someone making extremely niche mathematical software or something equally arcane.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#70

Earlier quoted context omitted.

I would argue that the overclocker needs to know more about these details than most programmers, yes. (Overclockers actually tweak these values to maximize the performance of their computer). But any high-performance programmer needs to understand the RAS / CAS / PRE cycle, if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient. If you are accessing RAM randomly, you b…

I am all for knowing things just to know things...but why does a programmer need to know WHY the streaming of data is efficient?

Because fundamentally, the DRAM is a major component of the computer, just as CPU cores are a major component of the computer.

Now, I'd personally explain things in a far more simple manner than what was described in the PDF. Here are the facts that programmers need to know:

1. DRAM stores data in very tiny capacitors. These tiny capacitors have two properties: they run out of electricity in just 64ms. And second, they run out of electricity after a SINGLE read operation.

2. DRAM has a temporary location called "sense amplifiers" where data is stored during a refresh or a read. These sense amplifiers can hold data permanently.

3. This "temporary read" is called Row-open (or RAS). Reading from an already open row is called a Column-read (CAS). Sending the data back to DRAM proper is called Precharge (PRE). Remember, the sense amplifiers must be clear before they can read from a new row. (The old data in DRAM was destroyed when you read it with the RAS operation)

4. I guess there's a periodic refresh you should know about: instead of trying to fix all RAM every 64ms, you're supposed to do it in small chunks at a time. Every dozen microseconds, RAM will self-read / self-write to refresh another row. Don't be surprised if your memory-reads randomly stall out an extra few hundred nanoseconds because of this refresh.

The end. Not so hard, now is it?

--------

DRAM is faster when you stream, because you open a row once, fill out all the data in a row, and then send the row back to DRAM. In effect, you only have to do a bunch of "column" writes to sense amplifiers, as opposed to opening-and-closing a bunch of different rows.

----------

So yeah, programmers should know it because its really not that hard to learn :-) And if you start measuring your program at the nanosecond level, you'll actually see these effects and start to demand explanations.

-------------

EDIT: Hmmm... the more I think of it, the less its something "programmers" need to know and something "SysAdmins / DevOps need to know". An advanced Sys Admin can use these profiler tools to figure out whether they need that 6x Memory Channel computer or the 8x Memory channel computer on the next purchase.

Is your code memory-bound? Or is it CPU bound? Should you buy more cores? Should you buy more LRDIMMs for higher amounts of RAM? Or is your program latency-bound and actually benefits from the lower latency of RDIMMs or even UDIMMs ?

The programmers kinda don't make those decisions.

Post reply on HN