Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Performance optimization is hard because it's fundamentally a brute-force task

purplesyringa.moe

Post reply on HN
Print view View original

71–80 of 153 posts

Re: Performance optimization is hard because it's fundamentally a brute-force task

#71

Post by Mawr »

purplesyringa wrote: post #70
trhway wrote: post #44

>Performance optimization is hard because it’s fundamentally a brute-force task, and there’s nothing you can do about it. fundamentally disagree. First it is a building of a mental model of what happens, a kind of analysis stage, and then compare it to the mental model of how it should or could work or producing a more efficient algorithm/way of accomplishing the target task. When people try to brute-force, lets try…

The problem is that way too often, the model simply doesn't capture enough complexity to be applicable. This happens rarely during high-level optimization but is very common during microoptimization. You can build a model, and it will give you good enough results, but you won't be able to extract those last bits of performance you need to surpass SOTA.

Sure, but that's a different statement. "Micro performance optimizations are fundamentally brute force" - ok.

Re: Performance optimization is hard because it's fundamentally a brute-force task

#72

Post by infogulch »

jandrewrogers wrote: post #42
jmull wrote: post #13

> I dislike the “intuition doesn’t work, profile your code” mantra because it seemingly says profiling is a viable replacement for theoretical calculations, which it isn’t. This seems like a nonsensical statement to me. How could measuring be a substitute for thinking/analyzing/predicting/forming a plan? Measuring/profiling just means observing the system you want to optimize in a systematic way. You certainly won't…

I think many people have seen both sides of this in practice. I’ve seen engineers follow the profiler into a dead-end because they see nothing that stands out in the profiler or they don’t grok the essential nature of the code or system they are profiling. I’ve seen engineers with deep domain expertise consistently make accurate estimates of how a code changes will impact performance without ever using a profiler bec…

Optimize according to your mental model; profile to check that your mental model matches reality.

If you optimize infrequently, or haven't profiled code like this recently, or haven't profiled this specific codebase recently, then your mental model is probably due a refresh.

Re: Performance optimization is hard because it's fundamentally a brute-force task

#73

Post by [deleted] »

mystified5016 wrote: post #38
9rx wrote: post #28

Earlier quoted context omitted.

Well, he's saying that intuition does work... But does it really? If a problem area is so intuitively obvious, why would you introduce the problem in the first place? In reality, performance optimizations are usually needed where you least expect them. Which means that you can't get there intuitively. Hence, the suggestion of using profiling to help track down where the problem is instead.

Do you want to claim you've never written quick and ugly code to get something working to come back and fix it up later? Pretty much everyone I know will throw down an O(n^2) algorithm or whatever in their first pass and replace it with something more thought out once they have the time to think deeply about it. If you're fretting about optimization at every stage of development, you're really doing it wrong. This is…

[deleted]

Re: Performance optimization is hard because it's fundamentally a brute-force task

#74

Post by tmoertel »

hinkley wrote: post #63
tmoertel wrote: post #53

Earlier quoted context omitted.

I think that there is a different reason that an emphasis on simple code often results in faster systems. When you write simple code, you spend less time writing code. Therefore, you have more time left to invest in optimizing the very small subset of your overall system that actually matters. You didn't burn engineering resources for speed where it didn't matter.

Simplifying complex code often exposes optimization opportunities. Kernighan’s Law applies to performance as well as debugging.

Indeed!

Re: Performance optimization is hard because it's fundamentally a brute-force task

#75

Post by 9rx »

mystified5016 wrote: post #38
9rx wrote: post #28

Earlier quoted context omitted.

Well, he's saying that intuition does work... But does it really? If a problem area is so intuitively obvious, why would you introduce the problem in the first place? In reality, performance optimizations are usually needed where you least expect them. Which means that you can't get there intuitively. Hence, the suggestion of using profiling to help track down where the problem is instead.

Do you want to claim you've never written quick and ugly code to get something working to come back and fix it up later? Pretty much everyone I know will throw down an O(n^2) algorithm or whatever in their first pass and replace it with something more thought out once they have the time to think deeply about it. If you're fretting about optimization at every stage of development, you're really doing it wrong. This is…

Purposefully introducing underperforming code as a tradeoff to meet some other goal (faster delivery, for example) is something else entirely. It doesn't require intuition or profiling – at most requiring only memory of the choice. A fun aside, but not related to what we're talking about.

Re: Performance optimization is hard because it's fundamentally a brute-force task

#76

Post by necovek »

purplesyringa wrote: post #59
jmull wrote: post #13

> I dislike the “intuition doesn’t work, profile your code” mantra because it seemingly says profiling is a viable replacement for theoretical calculations, which it isn’t. This seems like a nonsensical statement to me. How could measuring be a substitute for thinking/analyzing/predicting/forming a plan? Measuring/profiling just means observing the system you want to optimize in a systematic way. You certainly won't…

You're right, I could've phrased that better. Profiling to find suboptimal code is perfectly fine. Then you need to figure out how to fix it. Many people don't understand how performance optimization works, so they blindly add caching, improve constant time by invoking more low-level methods, etc. This obviously doesn't work, yet intuitively (to those people, anyway) it should produce good results. That's why the man…

"Intuitively" literally means without having to learn something.

Adding caches or switching to lower-level calls is definitely something learned, and I wouldn't call it "intuitive".

What I think you are referring to is that sometimes, simply reading and understanding the code can tell you where the problem really is — still, my experience is that you want to measure the before and after to at least identify the general area you should be looking at, and more often than not, I could figure out what needs optimizing and how without having to get detailed profiles.

I did end up being surprised a few times, but it was mostly due to external code like buggy library implementations that didn't do the things they promised (eg. async library really synchronizing everything with a badly placed mutex).

At the same time, it's wrong to focus on a single case or a single profile (executing one code path in one set of circumstances), but what you are describing simply sounds like bad engineering — the fact that you can misuse the tool does not make the tool bad, it's still the engineer wielding it who's at fault.

Re: Performance optimization is hard because it's fundamentally a brute-force task

#77

Post by patrick451 »

mystified5016 wrote: post #38
9rx wrote: post #28

Earlier quoted context omitted.

Well, he's saying that intuition does work... But does it really? If a problem area is so intuitively obvious, why would you introduce the problem in the first place? In reality, performance optimizations are usually needed where you least expect them. Which means that you can't get there intuitively. Hence, the suggestion of using profiling to help track down where the problem is instead.

Do you want to claim you've never written quick and ugly code to get something working to come back and fix it up later? Pretty much everyone I know will throw down an O(n^2) algorithm or whatever in their first pass and replace it with something more thought out once they have the time to think deeply about it. If you're fretting about optimization at every stage of development, you're really doing it wrong. This is…

This only works if you don't need real time performance. For anything that does need to run in realtime, you have to be damn sure that whatever your initial approach is can be optimized to run at the required sample rate. Otherwise, you will be ripping out the entire design and starting over. You might as well optimize first.

Re: Performance optimization is hard because it's fundamentally a brute-force task

#78

Post by hinkley »

tmoertel wrote: post #74
hinkley wrote: post #63

Earlier quoted context omitted.

Simplifying complex code often exposes optimization opportunities. Kernighan’s Law applies to performance as well as debugging.

Indeed!

If you can’t make it faster, make it dumber.

Re: Performance optimization is hard because it's fundamentally a brute-force task

#79

Post by kaptainscarlet »

jandrewrogers wrote: post #42
jmull wrote: post #13

> I dislike the “intuition doesn’t work, profile your code” mantra because it seemingly says profiling is a viable replacement for theoretical calculations, which it isn’t. This seems like a nonsensical statement to me. How could measuring be a substitute for thinking/analyzing/predicting/forming a plan? Measuring/profiling just means observing the system you want to optimize in a systematic way. You certainly won't…

I think many people have seen both sides of this in practice. I’ve seen engineers follow the profiler into a dead-end because they see nothing that stands out in the profiler or they don’t grok the essential nature of the code or system they are profiling. I’ve seen engineers with deep domain expertise consistently make accurate estimates of how a code changes will impact performance without ever using a profiler bec…

A lot of the optimisation I do largely consists of a series of highly educated guesses and the resulting fixes are right in 99% percent of the cases.

Re: Performance optimization is hard because it's fundamentally a brute-force task

#80

Post by AaronAPU »

tikotus wrote: post #34

I once had this kind of body recovery/stress level measuring thingy on me for a few days, and a doctor would then analyze my health and such. I was under some stress those days and (according to the measurements) I wasn't recovering properly even during the nights. But then there was this one, long, flat, deep green curve in the middle of my work day. I checked from my VCS what I was doing during that period: I was o…

I spent 10 years straight doing C++ and assembly optimization. My work is still fun these days but that was probably the most enjoyable work of my career in terms of the actual day to day coding.

Code cleanup in general is the same for me, but it’s really hard to justify putting much time into that when running your own company solo.

Post reply on HN