What Every Programmer Should Know About Memory (2007) [pdf] • Hacker News Board

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#31

Post by saberience » Mon, Mar 04, 2019, 7:59 PM UTC

magoghm wrote: post #26

saberience wrote: post #23

Earlier quoted context omitted.

If I saw that code at my company, I'd ask why someone building a rest api for web-services had to iterate through a 10000*10000 2D array while calling a method of every single cell. This isn't a use case I've seen in my 10 year career thus far. Also, you have to understand that saving 1 or 2 milliseconds isn't important when compared to getting software built faster. For the typical use case, optimizing a for-loop li…

Were I work we have to process a few billion records per day. Knowing about how memory works and writing the code in a way that uses memory bandwidth efficiently, allows us to process all those records in smallish AWS instance instead of having to use Hadoop on a much more expensive cluster of instances.

Hadoop is for "big data." Big data is commonly defined as minimum of petabytes of data. That is, thousands of terabytes. Our ingest of new events to our cluster is upwards of 25B per day and we keep all that data in perpetuity.

Our hadoop cluster is relatively small at 5PB right now, but I would be impressed indeed if you were processing PB of data on a smallish AWS instance.

If you were considering using Hadoop for work that can be processed on a "smallish instance" then I'd suggest you maybe don't really understand big-data or the normal workloads for which Hadoop is intended. For example, we run ML models on our Hadoop cluster which take hours to run distributed over around 100 nodes. Let me know when this can be done on a "smallish instance."

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#32

Post by mbauman » Mon, Mar 04, 2019, 8:00 PM UTC

saberience wrote: post #19

mbauman wrote: post #17

Earlier quoted context omitted.

Indeed, there are a limited number of hard-hitting topics that are definitely must-haves: * Rough latency timings * Caching * Prefetching * Sequential vs. random access * N-dimensional layouts (row/column major and arbitrarily strided) * Design of cache-oblivious algorithms * SIMD-able access patterns * False sharing * Instruction cache & code size * Branch prediction and speculative execution I'd be curious to hear…

Again, you say these are "must haves." But in the real world of software, none of these would make any difference to most of the software being built today, much of which is being build in higher level languages for web applications/web sites/microservices etc. Let's say someone is building a micro-service in C#.Net. Why would any of this stuff matter to them? The company cares about features and moving forward quick…

Sure, you don't really need to know any of this stuff unless you're actually needing to optimize code beyond the lowest hanging fruit.

My list presupposes you've hit a wall and need the best performance you can get. That's not always the case, but I certainly wouldn't say that these optimizations don't make a difference in the "real world of softare."

False sharing alone can be the difference between a 16x parallelization speedup and a 1000x+ slowdown over the naive serial algorithm. 16x can be the difference between actionable results tomorrow or three weeks from today... and if you didn't know about it, the 1000x+ slowdown would be otherwise inscrutable.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#33

Post by netmonk » Mon, Mar 04, 2019, 8:07 PM UTC

kevstev wrote: post #14

There are some salty comments here, but I think the context is important. This paper passed across my desk in early 2008 when I was doing HFT stuff. It might be a bit of a stretch to say that the reason people are taught about cache lines in most CS programs is because of this paper, but at the time this paper was written, this was really specialized knowledge and groundbreaking to most software developers. This woul…

I really do agree with you. HAving done HFT for some years, this paper was crucial when it was running on linux system. Now FPGA took over the field. Different kind of techs.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#34

Post by 3JPLW » Mon, Mar 04, 2019, 8:08 PM UTC

dragontamer wrote: post #18

mbauman wrote: post #17

Earlier quoted context omitted.

Indeed, there are a limited number of hard-hitting topics that are definitely must-haves: * Rough latency timings * Caching * Prefetching * Sequential vs. random access * N-dimensional layouts (row/column major and arbitrarily strided) * Design of cache-oblivious algorithms * SIMD-able access patterns * False sharing * Instruction cache & code size * Branch prediction and speculative execution I'd be curious to hear…

Good list. I'd add "Virtual Memory" to that list. In particular, the TLB cache, memory pages (4kB, 2MB "Large Pages", 2GB "Huge Pages). Although x86 specific, I'd also add x86-64 has 48-bit physical pointers: the top 16-bits are basically ignored by the current virtual memory system. I dunno if the whole Page Directory / Directory Tables / etc. etc. needs to be fully explained, but programmers should have an overall…

I don't think you need to explain the intricacies of the MESI protocol — just explaining the fact that caches need to be consistent is quite sufficient. Perhaps throw in why they must be consistent. It then becomes clear that the cores need to communicate (somehow) to maintain this consistency if they're touching data within the same cache line.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#35

Post by Chlorus » Mon, Mar 04, 2019, 8:09 PM UTC

mhh__ wrote: post #28

asaddhamani wrote: post #9

Earlier quoted context omitted.

A _lot_ more. I don't see how most programmers would benefit from reading 100+ pages of low-level details about memory.

Because knowing your craft is important? Besides, if you don't find this interesting why become a programmer in the first place

There's a lot to learn about this craft, and people have to prioritize - knowing algorithms & data structures is more immediately useful compared to, say, knowing what scratchpad memory is. If I spent my time learning every detail about every system underpinning every abstraction, I would literally be 70 years old by the time I started writing code.

> Besides, if you don't find this interesting why become a programmer in the first place

Who is saying it's not interesting? We're arguing that it's not fundamentally vital knowledge to know the difference in RAS & CAS latency for SDRAM for most programmers.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#36

Post by gdy » Mon, Mar 04, 2019, 8:09 PM UTC

saberience wrote: post #8

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

You should get acquainted with 'What Every Computer Scientist Should Know About Floating-Point Arithmetic' after which similar articles are named.

I'd bet you would suggest to rename it too.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#37

Post by netmonk » Mon, Mar 04, 2019, 8:10 PM UTC

oarabbus_ wrote: post #20

twtw wrote: post #4

Earlier quoted context omitted.

If you are objecting to "every," then sure, not every programmer needs to know anything about memory. You can program without knowing that there is such a thing. But it's a fun title, and an excellent resource.

No, he's right. It's more than simply semantics. A better title would be "an extremely overwhelming of programmers don't need to know THIS much about memory"

This is what people says when they have a dataset of 100 entries(<100mb) on their developpment laptop. How funny they look when production guys show them how their lame piece of code doesn't scale up with billion of entries in the production platform...But they dont care cause they are developpers, (or so called). Until the production guy rewrite their code and have them fired for incompetence (true story).

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#38

Post by amelius » Mon, Mar 04, 2019, 8:11 PM UTC

saberience wrote: post #8

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

I'm sure some programmers don't need to know this. For example an Erlang or Haskell programmer can't do anything about memory layout anyway, so this knowledge would be of no immediate practical use.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#39

Post by Chlorus » Mon, Mar 04, 2019, 8:16 PM UTC

netmonk wrote: post #37

oarabbus_ wrote: post #20

Earlier quoted context omitted.

No, he's right. It's more than simply semantics. A better title would be "an extremely overwhelming of programmers don't need to know THIS much about memory"

This is what people says when they have a dataset of 100 entries(<100mb) on their developpment laptop. How funny they look when production guys show them how their lame piece of code doesn't scale up with billion of entries in the production platform...But they dont care cause they are developpers, (or so called). Until the production guy rewrite their code and have them fired for incompetence (true story).

> How funny they look when production guys show them how their lame piece of code doesn't scale up with billion of entries in the production platform...But they dont care cause they are developpers, (or so called)

Why did the production guy even need them in the first place? Why were they hired?

> (true story). And that production guy's name was Albert Einstein.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#40

Post by bogomipz » Mon, Mar 04, 2019, 8:16 PM UTC

dragontamer wrote: post #21

saberience wrote: post #19

Earlier quoted context omitted.

Again, you say these are "must haves." But in the real world of software, none of these would make any difference to most of the software being built today, much of which is being build in higher level languages for web applications/web sites/microservices etc. Let's say someone is building a micro-service in C#.Net. Why would any of this stuff matter to them? The company cares about features and moving forward quick…

> I'm not saying it's not interesting, but it's not going to change how I write my for-loops and if statements or make calls to BigTable, etc. You might be surprised at how much changing your for loops can be when accessing matrix data. for(int i=0; i Test that out, and you'll see a major performance improvement. ----------- If someone is writing high performance code, the #1 goal is to be able to read your profiler'…

Can you elaborate why the second is faster in the general case? Is this an issue of row major vs column major orientation? Also does your example also hold in the absence of SIMD? Thanks.