Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Every Programmer Should Know About Memory (2007) [pdf]

akkadia.org

Post reply on HN
Print view View original

21–30 of 101 posts

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#21

Post by dragontamer »

saberience wrote: post #19
mbauman wrote: post #17

Earlier quoted context omitted.

Indeed, there are a limited number of hard-hitting topics that are definitely must-haves: * Rough latency timings * Caching * Prefetching * Sequential vs. random access * N-dimensional layouts (row/column major and arbitrarily strided) * Design of cache-oblivious algorithms * SIMD-able access patterns * False sharing * Instruction cache & code size * Branch prediction and speculative execution I'd be curious to hear…

Again, you say these are "must haves." But in the real world of software, none of these would make any difference to most of the software being built today, much of which is being build in higher level languages for web applications/web sites/microservices etc. Let's say someone is building a micro-service in C#.Net. Why would any of this stuff matter to them? The company cares about features and moving forward quick…

> I'm not saying it's not interesting, but it's not going to change how I write my for-loops and if statements or make calls to BigTable, etc.

You might be surprised at how much changing your for loops can be when accessing matrix data.

    for(int i=0; i
Test that out, and you'll see a major performance improvement.

-----------

If someone is writing high performance code, the #1 goal is to be able to read your profiler's output. If you can't read what the profiler says (cache hits, TLB hits, memory stalls, etc. etc.) then you can't make sense of the data.

The PDF goes deeper than I'd personally go, but there's some concepts here that are absolutely necessary if you actually want to understand what a decent profiler gives you these days.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#22

Post by 0815test »

dragontamer wrote: post #15
saberience wrote: post #8

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

I would argue that the overclocker needs to know more about these details than most programmers, yes. (Overclockers actually tweak these values to maximize the performance of their computer). But any high-performance programmer needs to understand the RAS / CAS / PRE cycle, if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient. If you are accessing RAM randomly, you b…

So, RAM is the new tape, cache is the new RAM?

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#23

Post by saberience »

dragontamer wrote: post #21
saberience wrote: post #19

Earlier quoted context omitted.

Again, you say these are "must haves." But in the real world of software, none of these would make any difference to most of the software being built today, much of which is being build in higher level languages for web applications/web sites/microservices etc. Let's say someone is building a micro-service in C#.Net. Why would any of this stuff matter to them? The company cares about features and moving forward quick…

> I'm not saying it's not interesting, but it's not going to change how I write my for-loops and if statements or make calls to BigTable, etc. You might be surprised at how much changing your for loops can be when accessing matrix data. for(int i=0; i Test that out, and you'll see a major performance improvement. ----------- If someone is writing high performance code, the #1 goal is to be able to read your profiler'…

If I saw that code at my company, I'd ask why someone building a rest api for web-services had to iterate through a 10000*10000 2D array while calling a method of every single cell. This isn't a use case I've seen in my 10 year career thus far.

Also, you have to understand that saving 1 or 2 milliseconds isn't important when compared to getting software built faster. For the typical use case, optimizing a for-loop like this for in-memory data isn't going to change the total performance of your program by even 0.5%. This is what so many people just don't understand. For modern web based applications, 99.9% of the time spent is doing IO, either hard-drive, data storage, or across the network. Optimizing those calls is FAR more important, than wasting time trying to save 1 millisecond in a for-loop which is iterating over 100 in-memory objects.

This is where the understanding that the practical reality of building software for a real business is very different to theoretical examples drummed up by computer scientists on discussion boards.

I see so many engineers trying to make their in-memory code faster by parallizing it, saving (if they're lucky) 1 or 2 milliseconds, but making their code far less maintainable and potentially introducing bugs, while the users wouldn't notice any practical difference in the app. Meanwhile, the app really does feel slow, because some numb-nuts made a call to some api and blocked the ui thread.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#24

Post by a1369209993 »

dragontamer wrote: post #15
saberience wrote: post #8

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

I would argue that the overclocker needs to know more about these details than most programmers, yes. (Overclockers actually tweak these values to maximize the performance of their computer). But any high-performance programmer needs to understand the RAS / CAS / PRE cycle, if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient. If you are accessing RAM randomly, you b…

> if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient.

I think what they're saying is that 99.9% of programmers don't need to know why, they just need to know that streaming access is 2-3x faster than random even without any memory stalls.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#25

Post by WallWextra »

saberience wrote: post #8

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

Mazel tov. There are also people who went to "Principal" without being able to program at all.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#26

Post by magoghm »

saberience wrote: post #23
dragontamer wrote: post #21

Earlier quoted context omitted.

> I'm not saying it's not interesting, but it's not going to change how I write my for-loops and if statements or make calls to BigTable, etc. You might be surprised at how much changing your for loops can be when accessing matrix data. for(int i=0; i Test that out, and you'll see a major performance improvement. ----------- If someone is writing high performance code, the #1 goal is to be able to read your profiler'…

If I saw that code at my company, I'd ask why someone building a rest api for web-services had to iterate through a 10000*10000 2D array while calling a method of every single cell. This isn't a use case I've seen in my 10 year career thus far. Also, you have to understand that saving 1 or 2 milliseconds isn't important when compared to getting software built faster. For the typical use case, optimizing a for-loop li…

Were I work we have to process a few billion records per day. Knowing about how memory works and writing the code in a way that uses memory bandwidth efficiently, allows us to process all those records in smallish AWS instance instead of having to use Hadoop on a much more expensive cluster of instances.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#28

Post by mhh__ »

asaddhamani wrote: post #9
saagarjha wrote: post #6

Earlier quoted context omitted.

"A bit more than what most programmers need to know about memory, but would be nice if they read anyways"?

A _lot_ more. I don't see how most programmers would benefit from reading 100+ pages of low-level details about memory.

Because knowing your craft is important? Besides, if you don't find this interesting why become a programmer in the first place

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#29

Post by jawnv6 »

dragontamer wrote: post #18
mbauman wrote: post #17

Earlier quoted context omitted.

Indeed, there are a limited number of hard-hitting topics that are definitely must-haves: * Rough latency timings * Caching * Prefetching * Sequential vs. random access * N-dimensional layouts (row/column major and arbitrarily strided) * Design of cache-oblivious algorithms * SIMD-able access patterns * False sharing * Instruction cache & code size * Branch prediction and speculative execution I'd be curious to hear…

Good list. I'd add "Virtual Memory" to that list. In particular, the TLB cache, memory pages (4kB, 2MB "Large Pages", 2GB "Huge Pages). Although x86 specific, I'd also add x86-64 has 48-bit physical pointers: the top 16-bits are basically ignored by the current virtual memory system. I dunno if the whole Page Directory / Directory Tables / etc. etc. needs to be fully explained, but programmers should have an overall…

how is MESI "an abstraction"? why is it the only model proffered instead of MESIF/MOESI?

and "the top 16-bits are basically ignored" is a funny way to spell "general protection exception on linear memory reference in non-canonical space" but sure, guess we're just handwaving here

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#30

Post by nestorD »

CalChris wrote: post #2

I wish Ulrich Drepper (thank you, Mr. Drepper) would update this with a section on Row Hammer and also Spectre and Meltdown . Programmer's need to know about memory because of these exploits, more so with the latter two in order avoid creating exploitable gadgets. But then I also think that What Every Computer Scientist Should Know About Floating-Point Arithmetic should be updated to include UNUMs. I don't think that…

What Every Computer Scientist Should Know About Floating-Point Arithmetic is great because it makes people aware of problems with the current system. Adding UNUMs would be suggesting the devil you don't know over the devil you know.

However, and I can't remember if it is alreaddy in the book, a section on compensated summation / dot product would give strong time tested tools to attack the problem from within the current framework.

Post reply on HN