Live data from Hacker News

Memory Bandwidth Napkin Math

forrestthewoods.com

11–20 of 56 posts

Re: Memory Bandwidth Napkin Math

#11
post #5

It's funny, I had this conversation last week as the coworker, but didn't actually know how to go about calculating this without running tests. It does seem like there are classes of problems which are completely bandwidth-limited. I've heard this is the next area of expansion for hardware tech, but I haven't seen much yet.

> I've heard this is the next area of expansion for hardware tech, but I haven't seen much yet.

This has been an continuously active area of hardware design since the 1960s (consider Cray's work on the 6600 and his later work at his own company). The whole HPC world has to obsess on this issue.

Re: Memory Bandwidth Napkin Math

#14
post #6

I’ve been fascinated by the 'napkin math' topic recently, but felt a need for a way to routinely practise. It’s an acquired skill once it’s effortless to do the order of magnitude calculations in a meeting, or zipping through possible solutions on a whiteboard (what I imagine Jeff Dean does). Will it be fast enough? How much will it cost? Does the benchmarked performance match the order of magnitude we’d expect? To p…

The two big things are a way to conceptualize your problem in a straightforward way (hard) and to learn to rapidly do adequate (not precise) arithmetic in your head (easy).

Wait, the math is easy? Sure, if all you are concerned with is the right, not correct answer. Back when people used slide rules this was common, but now when you do it it seems to weird some people out. You should develop this skill anyway because when you see an answer you should be able to tell at a glance if it's probably right or almost certainly wrong.

Why its simple: the first part is just to keep track of the order of magnitude.* People who used slide rules always had to to do this, and it's quick to pick up and pretty easy to do once you're used to it.

Second is just to know a few common fractions and be comfortable rounding intermediate results to convenient amounts (if you have "86" you might round it to 81 if you're dividing into thirds or ninths, or 88 if its by 11, or 80 or 90 if you care about 10x and would prefer your error to be a "too small" or "too big".

Third is to understand those error bars above, and, as you do when you work by floating point, avoid dealing with incommensurate numbers (this factor is so tiny I'll just ignore it).

When you get good at this you'll usually be within a few percent of the actual answer, which is usually enough to decide if it's worth actually calculating the answer or not.

Example: I remember a discussion a few years ago where we were trying to figure out if we could fit our product into a certain volume. As we discussed the parameters, I and a colleague simultaneously said "360 micrograms" (density was 1 so g = ml). The calculator welder beavered on and a few seconds later triumphantly said "357 mg". Great, 357 was more accurate than 360, but about right, but it was clear he was madly off in magnitude. He wanted to believe his calculator, and checked his work while the rest of us moved on.

* Unless you're a physicist in which case within a few orders of magnitude is probably OK, or a cosmologist in which case all you care about is 10^0

Re: Memory Bandwidth Napkin Math

#16
post #6

I’ve been fascinated by the 'napkin math' topic recently, but felt a need for a way to routinely practise. It’s an acquired skill once it’s effortless to do the order of magnitude calculations in a meeting, or zipping through possible solutions on a whiteboard (what I imagine Jeff Dean does). Will it be fast enough? How much will it cost? Does the benchmarked performance match the order of magnitude we’d expect? To p…

Thanks for making this list. And WOW!. What an impressive list of books You have read. I have Thinking Fast and Slow but haven't gotten around to reading it.

Re: Memory Bandwidth Napkin Math

#17
Yep. Most programmers today don't realize that "R" in "random access memory" can turn your RAM into a pumpkin very easily, and reduce bandwidth to below what you can get reading sequentially _from a budget SSD_.

Another number to put things into perspective: at 4GHz, 60ns is _240_ cycles. So every time you feel like you don't care about cache locality, try laboriously counting to 240.

Re: Memory Bandwidth Napkin Math

#20
> Let this sink in. Random access into the cache has comparable performance to sequential access from RAM. The drop off from sub-L1 16 KB to L2-sized 256 KB is 2x or less.

> I think this has profound implications.

I think I agree.

Can anyone here posit a theory why this is true? Is this a consequence of all the stream processing work in recent generations of processor? Or something else?

Is he saying that pointer chasing even when the values are in cache is the culprit?

Post reply on HN