The lack of discipline doesn't scale even more than discipline.
Not making a mess of it is the most challenging task we face when working on complex or even simple problems. A disciplined approach is essential to not making a mess of things.
Discipline Doesn’t Scale
21–30 of 173 posts
Re: Discipline Doesn’t Scale
#22I hate these articles that feature a programmer who works within a certain domain and assume all programming tasks are similar. Yes, it would appear there is an anti-pattern of programmers that feel like going lower level for certain things matters. It doesn’t always. Programming computers has multiple levels of abstraction for solving different problems. But there’s also this idea that the magical compiler does good…
> But there’s also this idea that the magical compiler does good enough abd there’s no reason to challenge the assembler. But there are programming problems where that is in fact essential. Citation needed. I've seen people claiming this for years, and I've yet to see a single case where handwritten assembler actually did better than spending the same amount of effort on speeding up the compiled program (e.g. taking…
> Citation needed. I've seen people claiming this for years, and I've yet to see a single case where handwritten assembler actually did better than spending the same amount of effort on speeding up the compiled program (e.g. taking 5 minutes to actually set the right target architecture).
Take some time looking at established open source projects where IBM has heavily invested in optimizing for the POWER architecture. I'm thinking things like glibc, gmp, Golang, openssl, ... The results they get from their hand rolled assembly far exceeds what gcc/llvm spit out. (At least, it did 2-3 years ago.)
Back when the Linux Technology Center (LTC) was a thing, I was fortunate enough to meet many of the individuals working on these projects. They are all wizards. Brilliant.
One of them retired recently and went on to start pveclib to make some of these optimizations in number crunching more accessible to others: https://github.com/open-power-sdk/pveclib
I was once in his office asking about assembly instructions for SHA-3 (a project that sadly didn't go very far) and he could quote useful scalar/vector instructions to me and their timing/latency stats faster than I could find them in the manual.
Re: Discipline Doesn’t Scale
#23Over a decade ago I used to argue this with the C++ committee people. Back then, they were not concerned about memory safety; they were off trying to do too much with templates. The C++ people didn't get serious about safety until Rust came along and started looking like a threat. Now they're trying to kludge Rust features into C++ by papering over the unsafe stuff with templates, with some success. But the rot under…
I really think lock-free concurrency models are the long term best bet. Locks and shared memory scale pretty badly, on top of being super hard to get right.
Re: Discipline Doesn’t Scale
#24I hate these articles that feature a programmer who works within a certain domain and assume all programming tasks are similar. Yes, it would appear there is an anti-pattern of programmers that feel like going lower level for certain things matters. It doesn’t always. Programming computers has multiple levels of abstraction for solving different problems. But there’s also this idea that the magical compiler does good…
> But there’s also this idea that the magical compiler does good enough abd there’s no reason to challenge the assembler. But there are programming problems where that is in fact essential. Citation needed. I've seen people claiming this for years, and I've yet to see a single case where handwritten assembler actually did better than spending the same amount of effort on speeding up the compiled program (e.g. taking…
> I've yet to see a single case where handwritten assembler actually did better than spending the same amount of effort on speeding up the compiled program (e.g. taking 5 minutes to actually set the right target architecture).
C++ with intrinsics can be very close to assembly. Compilers allocate register and do their magic with the rest of the code, but the real workload translates into assembly almost line-to-instruction. Takes longer than 5 minutes to do, these instruction sets are less than straightforward, but I think 80% of real-world performance sensitive compute-bound code is written like that (the other 20% being assembly).
> that will actually tell you what's going on
When I work on performance critical pieces, I already know what's going on. I design my code from the start to minimize cache misses and branches. The profiling tool I find most useful is sampling profiler. Then I either read assembly and try to do better, or rework higher-level logic or data structures.
Re: Discipline Doesn’t Scale
#25I hate these articles that feature a programmer who works within a certain domain and assume all programming tasks are similar. Yes, it would appear there is an anti-pattern of programmers that feel like going lower level for certain things matters. It doesn’t always. Programming computers has multiple levels of abstraction for solving different problems. But there’s also this idea that the magical compiler does good…
> But there’s also this idea that the magical compiler does good enough abd there’s no reason to challenge the assembler. But there are programming problems where that is in fact essential. Citation needed. I've seen people claiming this for years, and I've yet to see a single case where handwritten assembler actually did better than spending the same amount of effort on speeding up the compiled program (e.g. taking…
Branch mispredictions and cache misses might be hard to see in C and assembly, but in interpreted or JITted languages it's borderline impossible to reason about those things. Even compiled languages have trouble there if you cant control memory layout.
Re: Discipline Doesn’t Scale
#26Over a decade ago I used to argue this with the C++ committee people. Back then, they were not concerned about memory safety; they were off trying to do too much with templates. The C++ people didn't get serious about safety until Rust came along and started looking like a threat. Now they're trying to kludge Rust features into C++ by papering over the unsafe stuff with templates, with some success. But the rot under…
I really think lock-free concurrency models are the long term best bet. Locks and shared memory scale pretty badly, on top of being super hard to get right.
I like lockless synchronization, but livelock can also be an annoying problem.
Re: Discipline Doesn’t Scale
#27I hate these articles that feature a programmer who works within a certain domain and assume all programming tasks are similar. Yes, it would appear there is an anti-pattern of programmers that feel like going lower level for certain things matters. It doesn’t always. Programming computers has multiple levels of abstraction for solving different problems. But there’s also this idea that the magical compiler does good…
> Writing great asm or C is as challenging as writing great css I would argue that writing great asm that is more efficient than your idiomatic C code is going to be a very very challenging task on modern non-embedded CPUs. The compiler definitely knows more than I do about how the internal architecture works and which instructions to reorder to prevent stalls etc
> writing great asm that is more efficient than your idiomatic C code is going to be a very very challenging task
SIMD intrinsics are not idiomatic C, they are idiomatic assembly.
Here's an example when they improved by a factor of 3.6: https://stackoverflow.com/a/63759887/126995 That's not an unusual outcome.
Re: Discipline Doesn’t Scale
#28> And yet, for some people the problem with software isn’t a lack of automation but a lack of discipline. Software would be better if only people knew the rules, honoured them, and slowed themselves down so that instead of cutting corners they just chose to ignore important business milestones instead.
Who is arguing this?
Re: Discipline Doesn’t Scale
#29Over a decade ago I used to argue this with the C++ committee people. Back then, they were not concerned about memory safety; they were off trying to do too much with templates. The C++ people didn't get serious about safety until Rust came along and started looking like a threat. Now they're trying to kludge Rust features into C++ by papering over the unsafe stuff with templates, with some success. But the rot under…
I really think lock-free concurrency models are the long term best bet. Locks and shared memory scale pretty badly, on top of being super hard to get right.
Lock free algorithms and data structures are even harder to get right! And most of the canonical descriptions of them assume that you have a garbage collector and that memory allocation is a bounded operation. Neither of which is really true when you're working in constrained situations.
On top of that, locking is faster than most lock-free strategies unless you deal with high contention or deterimistic guarantees (90% of the latter is solved with bounded fifos).
Re: Discipline Doesn’t Scale
#30I hate these articles that feature a programmer who works within a certain domain and assume all programming tasks are similar. Yes, it would appear there is an anti-pattern of programmers that feel like going lower level for certain things matters. It doesn’t always. Programming computers has multiple levels of abstraction for solving different problems. But there’s also this idea that the magical compiler does good…
I hate these articles that feature a programmer who works within a certain domain and assume all programming tasks are similar.
Citation needed? Last I checked, Graham was working on a C64 Smalltalk80 VM as a side project, so I presume he's not completely ignorant of low-level development. And statements like "Your computer is not a fast PDP11" are aimed directly at low level development.