Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making Reasonable Use of Computer Resources

vfoley.xyz

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Making Reasonable Use of Computer Resources

#11

Post by hsn915 »

symmetricsaurus wrote: post #8

The author claims that making better use of the cache hierarchy and programming in a data oriented way will improve perceived performance for users. This isn't where we should start though. Much of the time bad performance comes from excessive I/O, bad algorithms and data structures (e. g. accidentally quadratic code), or misuse of third party libraries or services. Eliminating problems like this needs to come first…

A lot of poor performance in the wild comes from not batching operations that can be batched, thus increasing latency unnecessarily.

Utilization of the CPU's cache line is just one instance of this problem.

For example, imagine trying to parse a text file by literally reading one character at a time from the OS (literally issuing a 'read' command per character) vs reading the entire file upfront and then operating on the text in memory.

Even the if we assume OS does caching for disk data to minimize the actual number of reads from disk, you will still incurs at least an unnecessary cost from issuing all the sys calls (one per character).

The same can be said about, for example, issuing sql commands, and other things.

Re: Making Reasonable Use of Computer Resources

#12

Post by hsn915 »

vzaliva wrote: post #5

The SCIP quote the author tries to dispute is still very true. If you ask developers of a decently sized modern mature software system to list the main problems they are encountering, they will probably name complexity, legacy code, lack of documentation, bugs, and not the performance. As to making readable programs run fast, this is the job for compilers. Compilers should perform more and more sophisticated code tra…

Deferring responsibility to other people is not a suitable attitude for a professional software engineer.

There's a dangerous hidden assumption too in this mentality: the compiler is magic!

It's not.

The compiler mostly just does "micro optimizations" on the code.

The most trivial example to demonstrate this is looping on a grid by column vs by row. The by-column iteration will be slower due to bad utilization of the CPU cache, and no compiler will ever rearrange the loop. Not because compilers are being purposely stupid. There's just no way to prove that the semantics will stay the same (except in the most trivial situation).

Re: Making Reasonable Use of Computer Resources

#13

Post by II2II »

vzaliva wrote: post #5

The SCIP quote the author tries to dispute is still very true. If you ask developers of a decently sized modern mature software system to list the main problems they are encountering, they will probably name complexity, legacy code, lack of documentation, bugs, and not the performance. As to making readable programs run fast, this is the job for compilers. Compilers should perform more and more sophisticated code tra…

Your point that the application level programmer should not be burdened with details like CPU cache size is something the article's author may agree with. They mentioned that cramming 17 floats into a cache that can hold 16 is (premature) optimization.

On the other hand, the author is arguing that developers should take the presence of a CPU cache into consideration. The distinction is important. By failing to take the hardware into consideration it becomes much more difficult to solve performance issues down the road since an avenue to resolve it will likely involve rewriting disparate parts of code. Incidentally, this type of issue is unlikely to be addressed by optimizing compilers since it requires a knowledge of the runtime performance of disparate parts of code and may involve code that is not compiled at build time (e.g. the data structure may be passed to a third-party library).

Re: Making Reasonable Use of Computer Resources

#14

Post by II2II »

jcelerier wrote: post #7
vzaliva wrote: post #5

The SCIP quote the author tries to dispute is still very true. If you ask developers of a decently sized modern mature software system to list the main problems they are encountering, they will probably name complexity, legacy code, lack of documentation, bugs, and not the performance. As to making readable programs run fast, this is the job for compilers. Compilers should perform more and more sophisticated code tra…

> Compilers should perform more and more sophisticated code transformations Maybe but they don't, and my users asked for more performance yesterday (and the year before, and the year before that too, and likely the decade before that too). The problem with performance is that some people believe that it's a target that can be reached ; but the only acceptable target in most cases is "everything is instantaneous" - ot…

> and my users asked for more performance yesterday (and the year before, and the year before that too, and likely the decade before that too).

That made me think of the traditional response of throwing bigger and faster computers at a problem. The thing is, much of the performance of those bigger and faster computers comes from architectural optimizations that can only be fully exploited if the program takes advantage of them. In some cases, putting the code through an optimizing compiler may be enough. In other cases, the program needs to be explicitly designed to take advantage of those optimizations. (The cache is an okay example of this. There was a time when caches did not exist. The introduction of caches would have sped up some programs more than others, depending upon the program's design.)

Re: Making Reasonable Use of Computer Resources

#15

Post by kfprt »

symmetricsaurus wrote: post #8

The author claims that making better use of the cache hierarchy and programming in a data oriented way will improve perceived performance for users. This isn't where we should start though. Much of the time bad performance comes from excessive I/O, bad algorithms and data structures (e. g. accidentally quadratic code), or misuse of third party libraries or services. Eliminating problems like this needs to come first…

obligatory re. quadratic https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times...

Re: Making Reasonable Use of Computer Resources

#16

Post by wiz21c »

mlang23 wrote: post #9
vzaliva wrote: post #5

The SCIP quote the author tries to dispute is still very true. If you ask developers of a decently sized modern mature software system to list the main problems they are encountering, they will probably name complexity, legacy code, lack of documentation, bugs, and not the performance. As to making readable programs run fast, this is the job for compilers. Compilers should perform more and more sophisticated code tra…

I am working in HPC, and I beg to differ. Yes, compilers should do optimisation whereever possible, but that is only a small percentage of the story. If the programmer doesnt know which algorithms to choose, the compiler is unable to fix that. If the programmer does not understand the underlying architecture at least a bit, the compiler is also likely going to fail fixing that. Take one of the most basic programming…

yeah, but the more you optimize, the less your code is a way to communicate an idea on how/why (business wise) you did some things. I used to optimize some code. When I try to squeeze every CPU cycle, then I usually have to turn my algorithm upside down so that they "fit" correctly to the CPU architecture (cache, branch prediction, avoiding costly, instructions). In the end, algorithm cleverness, which is important to understand to read my code is totally muddied by optimization techniques here and there.

Having the right balance between performance and readability is tough...

Another example I cam across a while ago is matrix multiplication. See

https://en.wikipedia.org/wiki/Computational_complexity_of_ma...

The naive algorithm is super simple to understand and corresponds to the actual math definition. But the optimized algorithm, that's something totally different :-)

Re: Making Reasonable Use of Computer Resources

#17

Post by jameshart »

It’s interesting the author frames cache optimization as a way to make ‘reasonable use’ of computing resources.

But cache locality is only relevant to your code if you have a monopoly on a CPU core. There are likely hundreds of other processes running on a machine apart from yours and to make ‘reasonable’ use of computing resources your program needs to be effective and perform at even if those other processes preempt your code from running.

Making your code really good at using ALL the compute resources on a machine is not the same thing as making your code make reasonable use of computer resources.

Re: Making Reasonable Use of Computer Resources

#18

Post by rrss »

hsn915 wrote: post #12
vzaliva wrote: post #5

The SCIP quote the author tries to dispute is still very true. If you ask developers of a decently sized modern mature software system to list the main problems they are encountering, they will probably name complexity, legacy code, lack of documentation, bugs, and not the performance. As to making readable programs run fast, this is the job for compilers. Compilers should perform more and more sophisticated code tra…

Deferring responsibility to other people is not a suitable attitude for a professional software engineer. There's a dangerous hidden assumption too in this mentality: the compiler is magic! It's not. The compiler mostly just does "micro optimizations" on the code. The most trivial example to demonstrate this is looping on a grid by column vs by row. The by-column iteration will be slower due to bad utilization of the…

> the most trivial example to demonstrate this is looping on a grid by column vs by row. The by-column iteration will be slower due to bad utilization of the CPU cache, and no compiler will ever rearrange the loop

I think LLVM’s polyhedral optimization (Polly) probably will. I played with it a while back and found that if I handed it a naive matrix multiply with terrible locality, it was able to transform it to rearrange the iteration order and use tiling such that the naive w/ Polly ewas faster than any optimized version I could write. (my optimized versions probably weren’t great, but I tried the basics and got the normal speedups - Polly was just faster).

https://polly.llvm.org/ https://releases.llvm.org/12.0.1/tools/polly/docs/UsingPolly...

Re: Making Reasonable Use of Computer Resources

#19

Post by rrss »

jameshart wrote: post #17

It’s interesting the author frames cache optimization as a way to make ‘reasonable use’ of computing resources. But cache locality is only relevant to your code if you have a monopoly on a CPU core. There are likely hundreds of other processes running on a machine apart from yours and to make ‘reasonable’ use of computing resources your program needs to be effective and perform at even if those other processes preemp…

If one process is using only 4 bytes out of every 64 byte cache line fill, the rest of the line is wasted regardless of whether there are hundreds of other processes loading different lines into the various levels of cache. It’s not using more of the cache, just using the same lines (or fewer) more effectively.
Post reply on HN