171–180 of 392 posts
Admittedly clueless question: what does 'brrr' mean? (I'm not a software dev and idioms that may be obvious to others are unfamiliar to me.)
The trick to knowing we needed to improve it: profiling!
I'm the senior dev on my team, and whenever a new dev joined my team they would look at the codebase and go "ew, python2? Just use python3." That gave me a chance to explain the testing and refactoring cost that would come with changing python versions, and how the benefits to users would be almost zero. And then at some point one of the new juniors said, "hey, there's a lot of filesystem performance improvements and…
The reverse of this also happens: new team manager joins a team of 4-5 dev and goes "eww... a monolith, we'll write an MVP in 3 weeks with microservices, CQRS and all". Long story short, one year and a half passes and the mvp is still not finished, the architect leaves the company and some poor guys (from an outsourcing company) are still going at it with the same architecture.
Imagine that the intern didn't dare to ask such questions. a - he could've gone out thinking that performance doesn't matter. but it certainly does in a piece of code being used daily by thousands of devs. b - he could've thought that the simple implementation is faster but missed the fact that skip is implemented in assembly. c - he could've realized both but missed the why. and these failure scenarios are likely to…
> he could've thought that the simple implementation is faster but missed the fact that skip is implemented in assembly It’s not; the compiler is just fairly decent at transforming string manipulation routines.
oh my bad then, from this text it sounded like it was written in assembly.
I'm the senior dev on my team, and whenever a new dev joined my team they would look at the codebase and go "ew, python2? Just use python3." That gave me a chance to explain the testing and refactoring cost that would come with changing python versions, and how the benefits to users would be almost zero. And then at some point one of the new juniors said, "hey, there's a lot of filesystem performance improvements and…
Your experience sounds great, but it also sounds like we don't see that level of middle-ground pragmatism enough - it's either "no, we have to stay using PHP4.3.3 because it's what I know" or "we have to rebuild to cloud micro services to be able to scale infinitely without being restricted by schedules or budgets".
I'm the senior dev on my team, and whenever a new dev joined my team they would look at the codebase and go "ew, python2? Just use python3." That gave me a chance to explain the testing and refactoring cost that would come with changing python versions, and how the benefits to users would be almost zero. And then at some point one of the new juniors said, "hey, there's a lot of filesystem performance improvements and…
To be fair, it's easier to move to 3 now than X years ago. And there's more benefit.
Parts of the problem get worse as time goes on. (more code to convert, more complexity, more test cases, EOL for the python2 version of some lib, more user data, blah)
Parts of the solution get easier as time goes on. (easier syntax sugars, better testing frameworks, infrastructure abstracted easier, remove dead features, etc)
Parts of the desired solution become less relevant as time goes on. (Why not use golang or node or elixir, php and python are so dated!)...
Just because last year was a bad time for the upgrade doesn't mean today is. Knowing how to get these things done at the right time by the right people for the business is what separates a great engineering manager from one that is just "shipping features and bug fixes".
Earlier quoted context omitted.
It wasn’t taught to me. And, in my previous job I interviewed many dozen fresh grads. One of my questions was “How much slower is it to sum integers in a trivial linked list vs. a trivial array?” 90% answered “Umm... I don’t know. 2x?” When asked why, they all said “1 op to sum the int +1 op to traverse the pointer.” It was amazingly consistent.
What would be the kind answer you're looking for?
Imagine for the array it's 1 CPU instruction to load a value, 1 to load the next value, 1 to add them, and one to store the result, that would be 4 instructions per sum; ideally the array would stream into the CPU after a single main memory lookup delay up-front, and then be 4 instructions per pair, summed as fast as the CPU can loop.
The linked list at worst needs an imaginary 1 CPU instruction to load a value, 1 to load the pointer value, 1 to reference the pointer, a delay of 2 seconds to get that value from L1 cache - missed, it's not in cache - 240 seconds stalled waiting for main memory, 1 to add, 1 to store the result. Worst case, >240x slower.
The linked list is not guaranteed to be in contiguous memory, but it might be, so the cache might have the right data in it. The linked list is 50% data, 50% metadata, so the cache is half wasted / can hold half as much data, and if the linked list is coming in from a big memory read quickly, half the bandwidth is carrying pointer addresses not data, so the throughput is halved for that, too, and the processor cycles were already able to happen much faster than the main memory bus max speed. If it's not contiguous memory, you don't know in advance where it is to request all the right memory areas at once - not until you read sequentially to the last item pointer and find there are no more.
Maybe if they are both small, both in contiguous memory and go into Level 1 cache after a single main memory delay, it could be only ~2x time, but the more data there is overall, the more chance the linked list will bust the cache or be discontinuous in memory. And on the plain array side, it might be possible to speed up with SIMD/SSE instructions to spend fewer cycles adding and storing per element, which the linked list approach might not be amenable to at all[2], then best case might be ~4x slower, worst case ~500x slower.
[1] https://www.prowesscorp.com/computer-latency-at-a-human-scal...
[2] https://stackoverflow.com/questions/10930595/sse-instruction...
One of my job tasks was improving our custom string library. In VB! Sure InStr was slow: but we were trying to do "starts with" across very long strings. Sure, it worked, but a slightly smarter solution was 100 times faster. Upon finding it I went on a search, I found it written nearly identical 10 years before me, in a different version of a string library (hint, that one was faster). The trick to knowing we needed…
Admittedly clueless question: what does 'brrr' mean? (I'm not a software dev and idioms that may be obvious to others are unfamiliar to me.)
It originated from a meme of the Federal Reserve printing more monopoly money. "Printing machine goes brrrr". As they turn the crank ever faster making the printing machines smoke.
Senior Dev: Linked lists have very many more cache misses than do vectors, and the difference between hitting cache and hitting main memory is such a huge constant factor that for most reasonable list sizes it never makes sense to use a linked list. Use a vector. Checkmate, smug Lisp weenies.