Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Every Programmer Should Know About Memory (2007) [pdf]

akkadia.org

Post reply on HN
Print view View original

51–60 of 101 posts

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#51

Post by CalChris »

131012 wrote: post #10
CalChris wrote: post #2

I wish Ulrich Drepper (thank you, Mr. Drepper) would update this with a section on Row Hammer and also Spectre and Meltdown . Programmer's need to know about memory because of these exploits, more so with the latter two in order avoid creating exploitable gadgets. But then I also think that What Every Computer Scientist Should Know About Floating-Point Arithmetic should be updated to include UNUMs. I don't think that…

Thanks for bringing that up, I came to the comments section to see if some other details were outdated. Any other insights?

Well, I think hardware and software prefetch, speculation is a special form of prefetch, might be revisited. Software prefetch was at best a hopeful technology even in 2007 and is widely avoided today.

https://yarchive.net/comp/linux/software_prefetching.html

Something I don't think either Drepper or Hennessy + Patterson's books get across is memory banks from a programmer's perspective. How cache organization affects a program is explained well but how banks affect said same isn't. Construction yes. Visibility, no.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#52

Post by twtw »

saberience wrote: post #8

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

No, it isn't supposed to be ironic.

The title says "should," after all. Who says every programmer should knows these things about memory? Obviously not you. Ulrich Drepper does - I bet if you asked him, he would say everyone should know these things, but would concede that almost no programmers do and most programmers don't need to.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#53

Post by andyjpb »

saberience wrote: post #19
mbauman wrote: post #17

Earlier quoted context omitted.

Indeed, there are a limited number of hard-hitting topics that are definitely must-haves: * Rough latency timings * Caching * Prefetching * Sequential vs. random access * N-dimensional layouts (row/column major and arbitrarily strided) * Design of cache-oblivious algorithms * SIMD-able access patterns * False sharing * Instruction cache & code size * Branch prediction and speculative execution I'd be curious to hear…

Again, you say these are "must haves." But in the real world of software, none of these would make any difference to most of the software being built today, much of which is being build in higher level languages for web applications/web sites/microservices etc. Let's say someone is building a micro-service in C#.Net. Why would any of this stuff matter to them? The company cares about features and moving forward quick…

Having a certain amount of Mechanical Sympathy (https://mechanical-sympathy.blogspot.com/2011/07/why-mechani... ) is important for having a good intuitive feel for the appropriate way to implement something.

For example, an O(n^2) algorithm can often beat an O(n) one when n is "small". How do you get that "experienced" feel for which to choose when you're designing your program?

This is even important for people doing REST APIs, Web Services and Microservices. Milliseconds matter when the load gets beyond a certain point and your system ends up full of stragglers. How do you get that "experienced" feel for where that point might be and how far you can push a particular architecture without investing in a major rework?

The effect of tail latency on most web servers might surprise you:

http://latencytipoftheday.blogspot.com/

http://latencytipoftheday.blogspot.com/2014/06/latencytipoft...

TL;DR: most of your visitors will experience the bottom 1% of your performance curve on every page load.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#54

Post by esmi »

ontologiae wrote: post #42
saberience wrote: post #8

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

Every programmers should know something about memory, because software are becoming bloat [1] [2] because of a new generation of programmers who doesn't optimize their code. So, of course, you had a wonderful career, but it doesn't prove that you write efficient code [1] http://www.rntz.net/post/against-software-development.html [2] http://tonsky.me/blog/disenchantment/

It really all comes down to what variable one is trying to optimize. I noticed neither article mentioned money as a possible variable to optimize. Here is another take to consider:

https://www.joelonsoftware.com/2001/03/23/strategy-letter-iv...

"In 1993, given the cost of hard drives in those days, Microsoft Excel 5.0 took up about $36 worth of hard drive space. In 2000, given the cost of hard drives in 2000, Microsoft Excel 2000 takes up about $1.03 in hard drive space. In real terms, it’s almost like Excel is actually getting smaller!"

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#55

Post by [deleted] »

saberience wrote: post #8

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

[deleted]

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#56

Post by nickpsecurity »

saberience wrote: post #45
gdy wrote: post #36

Earlier quoted context omitted.

You should get acquainted with 'What Every Computer Scientist Should Know About Floating-Point Arithmetic' after which similar articles are named. I'd bet you would suggest to rename it too.

Computer Science is a different discipline to Software Engineering. I would argue that sure, if you want to be a PhD, academic, researcher etc, in Computer Science, then reading said article might be useful. For engineers working at companies whose goal is to get product to market faster, coding ruby on rails, javascript, making web apps, websites, it's far more useful knowing your web frameworks well and being able…

The referenced work talks about floating point errors that both computer scientists and field engineers need to avoid. Many folks in both camps screw up on floating point code without such guidance. So, at least that part is potentially helpful.

You just proved their point about how you'd reply, though.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#57

Post by dragontamer »

saberience wrote: post #31
magoghm wrote: post #26

Earlier quoted context omitted.

Were I work we have to process a few billion records per day. Knowing about how memory works and writing the code in a way that uses memory bandwidth efficiently, allows us to process all those records in smallish AWS instance instead of having to use Hadoop on a much more expensive cluster of instances.

Hadoop is for "big data." Big data is commonly defined as minimum of petabytes of data. That is, thousands of terabytes. Our ingest of new events to our cluster is upwards of 25B per day and we keep all that data in perpetuity. Our hadoop cluster is relatively small at 5PB right now, but I would be impressed indeed if you were processing PB of data on a smallish AWS instance. If you were considering using Hadoop for…

I don't work in big data. But here's what I think of your field anyway :-)

In general, you optimize software by starting from the slowest things first, and then working your way to the faster things when you run out of things to optimize. It seems like your problems are I/O bound, and therefore the bulk of optimizations can be made by simply optimizing your I/O (either through async calls, or better understanding of what the frameworks are actually doing, etc. etc.).

And that makes sense for sure.

The thing is: the next level of optimization is not CPU-optimization, but instead memory optimization. In fact, all CPU-based optimization starts at the Main-memory level. Why?

Because memory is slower than virtually everything inside of the main CPU Core.

Which means, memory-level optimization is a far more important skill than any other CPU-based optimization technique. Modern CPU Cores operate at 4GHz easily, but RAM only responds every 100ns on server systems (that's 400 CPU-cycles of RAM Latency!)

-----------

In effect, if there's one low-level optimization any higher-level programmer should know about, it is RAM optimization. Sure, there are CPU-optimizations (SIMD registers, L1 vs L2 cache, and more), but RAM access itself is hundreds of times slower than CPU-speeds and needs to be optimized before lower-level optimizations are sought.

Once RAM access are fully optimized, then you can finally reach for the CPU-core optimizations. Instruction-level parallelism, or SIMD Registers, or what-not.

RAM is the next slowest part of your system after I/O. As such, optimizing RAM access is the most logical next step forward when your programs are running poorly.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#58

Post by nickpsecurity »

saberience wrote: post #19
mbauman wrote: post #17

Earlier quoted context omitted.

Indeed, there are a limited number of hard-hitting topics that are definitely must-haves: * Rough latency timings * Caching * Prefetching * Sequential vs. random access * N-dimensional layouts (row/column major and arbitrarily strided) * Design of cache-oblivious algorithms * SIMD-able access patterns * False sharing * Instruction cache & code size * Branch prediction and speculative execution I'd be curious to hear…

Again, you say these are "must haves." But in the real world of software, none of these would make any difference to most of the software being built today, much of which is being build in higher level languages for web applications/web sites/microservices etc. Let's say someone is building a micro-service in C#.Net. Why would any of this stuff matter to them? The company cares about features and moving forward quick…

"Why would any of this stuff matter to them?"

Back when I started, both the BASIC and Common Lisp camps had heuristics on helping the compiler generate more efficient code. The reasons are similar to today: increase throughput for scaling or customer experience while simultaneously reducing what you spend on hardware and/or electricity. Lean, easy-to-manage setups can also sometimes let you afford extra personnel to build stuff faster.

There's definitely a cut-off where micro-optimizations wouldn't be necessary. A lot of efficiency gains are simple, though. Just gotta consistently use what you learn.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#59

Post by lxe »

saberience wrote: post #8

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, programmer would need to know about transistor configurations or…

I work in frontend after switching gears from embedded systems a few years back. Knowing some level of detail about how computers work is invaluable at all layers of the stack: I can make informed trade-offs between practical performance of code running on an actual computer and the cost of high-level language concerns and features.

Re: What Every Programmer Should Know About Memory (2007) [pdf]

#60

Post by esmi »

dragontamer wrote: post #21
saberience wrote: post #19

Earlier quoted context omitted.

Again, you say these are "must haves." But in the real world of software, none of these would make any difference to most of the software being built today, much of which is being build in higher level languages for web applications/web sites/microservices etc. Let's say someone is building a micro-service in C#.Net. Why would any of this stuff matter to them? The company cares about features and moving forward quick…

> I'm not saying it's not interesting, but it's not going to change how I write my for-loops and if statements or make calls to BigTable, etc. You might be surprised at how much changing your for loops can be when accessing matrix data. for(int i=0; i Test that out, and you'll see a major performance improvement. ----------- If someone is writing high performance code, the #1 goal is to be able to read your profiler'…

I have to admit, I was skeptical but my complier did pick SIMD for the second and not the first and it did make a huge difference.

  % cat t.c
  #include 
  #include 
  #include 

  double get_time()
  {
    struct timeval t;
    struct timezone tzp;
    gettimeofday(&t, &tzp);
    return t.tv_sec + t.tv_usec*1e-6;
  }

  int blah[SIZE][SIZE];
  int whatever() { static int i=0; return ++i; }

  int main()
  {
    double t0 = get_time();

    for(int i=0; i
2.8G Core i7 1600MHz DDR3
Post reply on HN