New Grad vs. Senior Dev • Hacker News Board

Re: New Grad vs. Senior Dev

#61

Post by TeMPOraL » Sat, Mar 28, 2020, 4:51 PM UTC

jakear wrote: post #50

Heh... reminds me of my first proper MS internship, when I too was responsible for speeding up some code, this time in the VS Code Go extension. This code was responsible for tokenization, so it affected pretty much every operation and ran on every edit. Important shit. Day 1: do some basic hoisting. O(n^3) => O(n^2). Tokenization times for a 10k line file go from ~15s to 500ms. Sweet. Days 2-30 [1]: ideate, develop,…

> But hey, all the people working in 500k line files must really love the couple seconds my month of toiling (more importantly, my month of not doing other, more impactful things) saved them.

This stuff matters. These couple seconds per operation may very well be a difference between being able to open a 100k+ LOC file in the same editor you're doing your other work in, vs. giving up in frustration and looking for something else that can handle large files. Large files happen surprisingly often (in particular: log files, data dumps, machine-generated code). A "month of toiling" like this may immediately enable new use cases.

Re: New Grad vs. Senior Dev

#62

Post by ericlippert » Sat, Mar 28, 2020, 4:52 PM UTC

jakear wrote: post #50

Heh... reminds me of my first proper MS internship, when I too was responsible for speeding up some code, this time in the VS Code Go extension. This code was responsible for tokenization, so it affected pretty much every operation and ran on every edit. Important shit. Day 1: do some basic hoisting. O(n^3) => O(n^2). Tokenization times for a 10k line file go from ~15s to 500ms. Sweet. Days 2-30 [1]: ideate, develop,…

One of the most important things you can do in perf analysis is to know when to stop looking for incremental improvement.

If this subject in particular interests you, we did a lot of work in the C# lexer/parser so that once the file is lexed, it only re-lexes the tokens which changed on every edit. It also does fun stuff like the syntax colourizer only runs on code that's actually on the screen. Getting every operation that depends on the lex/parse of code in the editor down to running in much less than 30ms so that it would not slow down keystrokes was a huge amount of work.

Re: New Grad vs. Senior Dev

#63

Post by alexis_fr » Sat, Mar 28, 2020, 4:54 PM UTC

dmoy wrote: post #42

TeMPOraL wrote: post #35

Earlier quoted context omitted.

It's not even that. You could have a normal hash table with a decent hashing function, and you'll still get beaten by a flat array for small n (hundreds, low thousands), because the array is contiguous in memory - so operations like search or moving stuff around after addition make extremely good use of CPU's cache.

> the array is contiguous in memory - so operations like search or moving stuff around after addition make extremely good use of CPU's cache Also - if I see someone try to use a linked list for an enormous data structure again.... Wow it does not scale worth crap because it turns out that the hardware is actually important, and contiguous memory is amazing.

> if I see someone use a linked list

Or a hashmap to prepare 3 variables to pass to Json serialization.

Re: New Grad vs. Senior Dev

#64

Post by asveikau » Sat, Mar 28, 2020, 4:55 PM UTC

saagarjha wrote: post #25

> The skipto method is a single x86 machine instruction. That’s not always a good thing, especially on modern hardware. And obviously, the “single instruction” doesn’t mean it’ll take bounded time to execute…

Tangentially to your point, here's something I haven't thought about much: when these instructions get an interrupt, I imagine they've updated (r|e)si and (r|e)di, (r|e)cx etc. to reflect where they are in their copy or scan loop. So if you get a page fault in the middle, then the kernel does enormous amounts of work in response to it, then resumes that single instruction, it resumes in the middle of the loop, not the start.

So to reiterate one aspect of your point, there might be lots of work, written in C, that occurs in response to the page fault in the middle of your "hardware-backed" single instruction. On top of all the other complexities of cache vs memory access etc. that make scanning and copying memory complicated no matter which way you do it.

But probably in the heyday of Visual Basic, and especially the DOS-based BASICs that preceded it which wouldn't have had virtual memory at all, all of this is less of a concern. The story takes place in a simpler time which serves as a plot device to better illustrate the point.

Re: New Grad vs. Senior Dev

#65

Post by corysama » Sat, Mar 28, 2020, 4:58 PM UTC

dimtion wrote: post #7

I find that the biggest misunderstanding happens because "new grads" (and I happen to be one) confuse _asymptotic complexity_ with actual complexity. I'm not sure sure why, but CS courses and interview questions mostly focus on _asymptotic complexity_ and usually forget to take into consideration the complexity for "little values of n". And funnily enough, in real life n never goes to infinity! In a strict sense big…

It’s because Big O is Computer Science. Cache effects are Software Engineering. Professors of CS do a fine job of teaching CS. They even briefly mention that there is a implicit constant factor k in O(k n log(n)) and then they never mention it again. They certainly don’t mention that k can easily vary by 128x between algos. AKA: 7 levels of a binary tree. Or that most of the data they will be dealing with in practice not only won’t be infinite, but will actually be less than 128 bytes. Or, that even with huge data and an proven-ideal O() algo, there is often 10x speed-up to be had with a hybrid algo like a b-tree instead of a binary tree. And, another 2-10x with SIMD vs scalar. 100x isn’t infinite, but it’s still counts.

So, grads listen to their CS professors and that’s what they know. It’s not until they get lectures from greybeard software engineers that they learn about the reality algos and not just the idealized algos.

Re: New Grad vs. Senior Dev

#66

Post by tyleo » Sat, Mar 28, 2020, 5:02 PM UTC

I see these senior vs non-senior engineer contrasts pop up a lot. I’m not a huge fan of them.

It seems that there is a spectrum of skills an engineer could excel at: programming, infrastructure, managing, planning, etc. I’ve known senior engineers who only excel at a particular skill. I’ve also known senior engineers who are moderately good at many but not particularly good at one.

In my experience the only difference between a senior and non-senior is that the senior’s distribution of skills makes them more effective.

I’ve also seen senior engineers change roles laterally and become more or less effective, yet still maintain the senior title.

Re: New Grad vs. Senior Dev

#67

Post by dilyevsky » Sat, Mar 28, 2020, 5:03 PM UTC

corysama wrote: post #65

dimtion wrote: post #7

I find that the biggest misunderstanding happens because "new grads" (and I happen to be one) confuse _asymptotic complexity_ with actual complexity. I'm not sure sure why, but CS courses and interview questions mostly focus on _asymptotic complexity_ and usually forget to take into consideration the complexity for "little values of n". And funnily enough, in real life n never goes to infinity! In a strict sense big…

It’s because Big O is Computer Science. Cache effects are Software Engineering. Professors of CS do a fine job of teaching CS. They even briefly mention that there is a implicit constant factor k in O(k n log(n)) and then they never mention it again. They certainly don’t mention that k can easily vary by 128x between algos. AKA: 7 levels of a binary tree. Or that most of the data they will be dealing with in practice…

Every good cs course has a section on cache aware algorithms. And i call bullshit that constant factor is not mentioned too

Re: New Grad vs. Senior Dev

#68

Post by RookyNumbas » Sat, Mar 28, 2020, 5:03 PM UTC

kevinventullo wrote: post #14

dimtion wrote: post #7

I find that the biggest misunderstanding happens because "new grads" (and I happen to be one) confuse _asymptotic complexity_ with actual complexity. I'm not sure sure why, but CS courses and interview questions mostly focus on _asymptotic complexity_ and usually forget to take into consideration the complexity for "little values of n". And funnily enough, in real life n never goes to infinity! In a strict sense big…

To be fair, in my experience it is often the case that asymptotic complexity is a good proxy for real-world performance, even for small values of n. Not always, but often. I think it's fine that the academic courses focus a bit more on what's better in theory than in practice, because there are always caveats to "in practice"; the person who writes the special-purpose genomics libraries was also once a new grad.

I like to start by thinking about cache locality and ensuring linear layout. Next focus on one-time, or minimal memory allocation. Then there are a bunch of small, systemic things you need to get right. After that you can start worrying about worst case big O scenarios.

Of course this depends on your language. A c programmer will have a different mental model than a python one.

Re: New Grad vs. Senior Dev

#69

Post by thedance » Sat, Mar 28, 2020, 5:04 PM UTC

vardump wrote: post #60

saagarjha wrote: post #25

> The skipto method is a single x86 machine instruction. That’s not always a good thing, especially on modern hardware. And obviously, the “single instruction” doesn’t mean it’ll take bounded time to execute…

Very much true. REP SCASB on 1994 (same year as the incident in the article) Intel Pentium would have taken 9 + 4 * n clock cycles [0][1] to go through a string. So scanning 4 chars long string takes 25 clock cycles. [0]: Agner's instruction tables page 123: https://www.agner.org/optimize/instruction_tables.pdf [1]: Plus one extra cycle to decode REP prefix, if previous instruction took 1 cycle.

It could be faster than some alternatives under some circumstances. A loop would probably be more code (== icache pressure) and would consume at least a register and a BTB entry.

Re: New Grad vs. Senior Dev

#70

Post by loosetypes » Sat, Mar 28, 2020, 5:05 PM UTC

dimtion wrote: post #7

I find that the biggest misunderstanding happens because "new grads" (and I happen to be one) confuse _asymptotic complexity_ with actual complexity. I'm not sure sure why, but CS courses and interview questions mostly focus on _asymptotic complexity_ and usually forget to take into consideration the complexity for "little values of n". And funnily enough, in real life n never goes to infinity! In a strict sense big…

Complexity yes, but I feel scalability is similarly a word with a misleadingly narrow connotation in practice.

Is an approach dependent on swathes of training data truly scalable if it doesn’t work for the first n attempts?