"but software development is not data structures and algorithms"
Green Lumber Fallacy in Software Engineering
121–130 of 238 posts
Re: Green Lumber Fallacy in Software Engineering
#122Earlier quoted context omitted.
The point there is standardized . If you apply a standardized test, then it becomes feasible to measure and argue the disparate impact. For all their failings, leetcode-style interviews are probably less culturally-sensitive than standardized IQ tests. They probably do have (unintentional) disparate impact, but this seems like a really hard thing to measure (and possibly correct for).
> If you apply a standardized test, then it becomes feasible to measure and argue the disparate impact. Yes, the existence of readily available statistics may make the unequal impact easier to show. > For all their failings, leetcode-style interviews are probably less culturally-sensitive than standardized IQ tests. “Culturally sensitive”, maybe, but they almost certainly have quite large unequal impacts adverse to p…
You are committing a corollary to the green lumber fallacy described by the article.
Just because code challenges are not relevant to job duties doesn’t mean the results are irrelevant to job performance. They are a proxy intelligence test. General intelligence is the best predictor of success in almost any role (not just software engineering).
This is what confounds people. They think the interviews are designed that way because they are supposed to be representative of the job — I don’t believe that’s the case. They are that way because they provide a strongly correlated signal of performance after hire, and big tech has decades of data for all different interview types. I’m very confident that if they had a better interview circuit that could be done in a ~day, they would be doing that. Obviously even at big tech, referrals and recommendations count for a lot.
While I don’t like the low recall, I do think that “invert this binary tree” probably has less bias than a quiz on technology, or a design conversation (that seems way more susceptible). Perhaps it has a bias for a particular kind of computer science education and thinking, but at least that’s not a protected class. I’m not seeing the age connection, but I could imagine e.g. a computer science education in different countries emphasizing different skills over years (and leading to some candidates with a leg up on the “tests”).
Re: Green Lumber Fallacy in Software Engineering
#123I used have a similar perspective as the author, that the leetcode style questions / data structure / algorithm stuff they ask in interviews but seems to have little to do with day-to-day corporate software development is all a big waste of everyone's time. I'm not sure I still do, though. It's true that this stuff rarely comes up and often doesn't matter for most real-world applications... but when it does, it matte…
When it comes up do you need to code it up in 20 minutes or is looking the algorithm up in a textbook ok? If all you want to test that sort of knowledge just ask the knowledge question. I don't need to memorize Floyd-Warshal and be able to code it on demand to be able to recognize when it's needed and look it up. I'm sure I remembered it back in school, haven't used it since, but if the day comes I'll look it up...
I do think there's a real place for interview questions where you identify algorithmic opportunities in real world problem scenarios quickly, but you also may be missing some of the incentives for testing the way it's currently tested:
Coding floyd-warshal importantly shows that you can code, and generally signals at least one of two things:
1. You studied hard for the interview, you're willing to do difficult and unpleasant intellectual work for rewards and do so in a structured enough manner to get results.
2. You have such a deep understanding of graphs and graph algorithms you were able to re-construct floyd-warshal in 45 minutes. This implies time will not be an issue if you see an opportunity for these problems, and you can probably handle much harder problems as well.
Filtering for these two traits might work very well depending on the hiring needs of a particular company.
Re: Green Lumber Fallacy in Software Engineering
#124Its shocking to me how people don't get bored of rehashing this same, tired opinion over and over and over again and discussing it with hundreds of comments. There is a blog post like this every other day on the front page of HN and it always generates the exact same discussion.
I have said this before: it's group therapy for entitled devs that feel they're entitled to oversized salaries just because they have tenure. Young grads accept the process as is. People that are either happy with their comp or comfortable putting in the work do not lament the process - they just accept it and proceed. Only devs that have been doing it a while, whose salaries have plateaued, that either never learned data structures or can't muster the will to review, bemoan the LC grind.
The most utterly absurd instantiation is the dev that's 100% sure that FAANG is going to fail soon because they're passing on people like him/her that are just so talented that they don't need to prove it with any assessment. There's also the MGTOW-esque subgroup that eschews such interviews and will proclaim it proudly, along with their salary.
Personally I'd be embarrassed if I was making a less than obscene dev salary, which is still upper middle class anywhere in the world, and all the while complaining that I deserve more for no added effort.
What's funny is there are other professions where the same sort of multimodal distribution in compensation exists (big law vs everyone else, competitive specialties vs general practice in medicine, IB and PE vs everyone else in finance) and I have never seen members of those communities complain as much as developers do.
Edit:
I have a friend that's trying to break into FAANG (as a DS) who I've been coaching. He has a stats PhD and basically no software training whatsoever, outside of R. Since DS loops still have LC questions it's been very hard for him. I have never heard him once express feelings of resentment over the process - he fails an interview and just goes back to practicing. He's also a first gen immigrant from a very poor part of the middle east. Admittedly it's hard to assign/attribute his perseverance to any particular thing but naive intuition dictates that some of it must be his humility - something that software as an industry seriously lacks.
Re: Green Lumber Fallacy in Software Engineering
#125I agree with the author, mostly, though I don't have as much of a problem with the deeply technical questions he's suggesting one shy away from -- my problem is with how the responses are handled by the interviewer. And as far as the whole "give them the actual work" approach, I love it -- I do it, and I stand by it, but don't abuse the candidates time in the process.
For the most part, I avoid the data structures-style questions. We have a limited time as interviewers and while these show off a candidates knowledge in areas we may need very rarely, I'd rather have someone who is productive at using what the framework we write code in provides. I stick with "when is it appropriate to use 'X' vs 'Y'" style questions but usually don't have to ask them; the candidate reveals that knowledge (or lack thereof) through other questions and I hate whiteboard interviews. I don't penalize the candidate for lacking knowledge they're never going to use on the job, but I like to know what areas they have deep understanding of. Before I ask these questions, I'm careful to point out "I don't care if you know the answer to this or not -- you might encounter that once in 10 years, here -- but if we do encounter it, it's helpful to know if you have a deeper understanding than others on the team." For the most part, though, it's a waste of time and I'd rather spend that time on the next two points.
My preference is to ask the candidate (very carefully) to provide a link to anything they've written that they can legally[0] share openly. My explanation includes all of the following: I am not interested in judging them on the code they send me -- in fact, I want their worst. I want the code that they don't care about, that they wrote in a hurry to solve some random problem that they had and that "just needed to work and do that one thing". I'll use that code as a starting point to discuss refactoring -- i.e. "What if you wanted to take this code and make it worthy of a product you'd sell". If there's absolutely nothing that they are comfortable[1] sharing, I ask them to take a few minutes and find a project in the framework we're interviewing for on GitHub that we can use for that purpose.
As for the "give them the actual work", we have a single-sheet project we give to every candidate, Junior, Mid or Senior. It's a "to do" app with basic requirements: Must use a database of some kind for storing/retrieving data (Sqlite is called out as an option, but anything relational is allowed), it must allow adding, marking complete, unmarking complete and deleting. We additional coach Senior/Mid developers to include things that would be important for a completely released application with examples (none of which are required), such as documentation, instructions for starting/setting it up, installers (with advice not to take this too far as that's one hell of a rabbit hole). Depending on the outcome of the "actual work", we might follow-up in a similar manner to ask them questions about their choices.
I've seen everything from a To Do app that used SignalR to perform its operations, allowing multiple users to watch the check-list get updated in real-time (complete with a WiX installer, OWIN hosted web service running as a Windows Service with everything wired up and a doc generation project[2]) to a simple HTML-template based solution that is done in four files. Of course, we check the usual sources for obvious evidence of whole-sale copying, but we're not terribly strict here, either (this point is made on the spec sheet -- it's such a simple app, it shouldn't require a lot of research/copy-pasta to build).
[0] The first time I did this, I left off that "legally" part and the candidate shared code that -- while not representing anything their employer would care about (basic library code that probably every programmer as written from time to time) and certainly not representing anything the company I worked for was interested in "stealing", it carried a copyright that they did not own. Unfortunately, it disqualified the candidate (not my choice in this case, but the company I was working for at the time had concerns that the employee might introduce liability we didn't want). And I don't want any candidate to think I'm asking them to break their contractual obligations in order to get a job -- we operate ethically and don't want to imply otherwise.
[1] I'm always careful to say "comfortable" so as to allow them to not have to feel bad about not having any code written "outside of work". I have met many software developers (albiet often in the Junior/mid-level skill levels) who simply don't care to write software outside of work. At the senior level, I've met several software developers who haven't written anything publicly shared in a while -- they're working too much at their current job, have families, and might not have the time to devote to things outside of that. Interviewing sucks and anything I can do to disarm the person and get them talking/excited will help me to evaluate them better.
[2] The dude really wanted the job. OK, I lied, the dude was me, but it was rare that I wouldn't include these things in an app I was writing for my current employer, so those parts were automatic (and easy at that point) for me -- it felt wrong not to include them. I used the documentation to explain my choices, mostly, which avoided a second technical interview.
Re: Green Lumber Fallacy in Software Engineering
#126I used have a similar perspective as the author, that the leetcode style questions / data structure / algorithm stuff they ask in interviews but seems to have little to do with day-to-day corporate software development is all a big waste of everyone's time. I'm not sure I still do, though. It's true that this stuff rarely comes up and often doesn't matter for most real-world applications... but when it does, it matte…
Over and over, I see code written by star professionals that you've heard of, reinventing the bubble sort.
I'm guilty of it myself.
> It's a bit like saying it's silly for pilots to have to know what to do in the case of engine failure or stall from memory and regularly practice in a simulator, because that stuff rarely happens and most of the time they're just using autopilot. That's true, it usually doesn't matter, but when it does... it matters a lot.
Well, we now know what happens when pilots forget the runaway stabilizer trim procedure (turn off the stab trim).
Re: Green Lumber Fallacy in Software Engineering
#127I used have a similar perspective as the author, that the leetcode style questions / data structure / algorithm stuff they ask in interviews but seems to have little to do with day-to-day corporate software development is all a big waste of everyone's time. I'm not sure I still do, though. It's true that this stuff rarely comes up and often doesn't matter for most real-world applications... but when it does, it matte…
When it comes up do you need to code it up in 20 minutes or is looking the algorithm up in a textbook ok? If all you want to test that sort of knowledge just ask the knowledge question. I don't need to memorize Floyd-Warshal and be able to code it on demand to be able to recognize when it's needed and look it up. I'm sure I remembered it back in school, haven't used it since, but if the day comes I'll look it up...
No post body was provided.
Re: Green Lumber Fallacy in Software Engineering
#128Earlier quoted context omitted.
When it comes up do you need to code it up in 20 minutes or is looking the algorithm up in a textbook ok? If all you want to test that sort of knowledge just ask the knowledge question. I don't need to memorize Floyd-Warshal and be able to code it on demand to be able to recognize when it's needed and look it up. I'm sure I remembered it back in school, haven't used it since, but if the day comes I'll look it up...
I do think there's a real place for interview questions where you identify algorithmic opportunities in real world problem scenarios quickly, but you also may be missing some of the incentives for testing the way it's currently tested: Coding floyd-warshal importantly shows that you can code, and generally signals at least one of two things: 1. You studied hard for the interview, you're willing to do difficult and un…
I would say:
1. You're fresh out of school or you have a lot of time on your hands.
2. You memorized some interview preparation book.
I do agree that a candidate that has shown some effort preparing for an interview is a positive. Other than that this seems like a silly game.
The flip side is the candidate that memorized the book, prepared like it's a test at school, and is useless on the job, can't operate at all outside that memorized domain, can't write the simplest bit of real world code.
Re: Green Lumber Fallacy in Software Engineering
#129I used have a similar perspective as the author, that the leetcode style questions / data structure / algorithm stuff they ask in interviews but seems to have little to do with day-to-day corporate software development is all a big waste of everyone's time. I'm not sure I still do, though. It's true that this stuff rarely comes up and often doesn't matter for most real-world applications... but when it does, it matte…
Over and over, I see code written by star professionals that you've heard of, reinventing the bubble sort. I'm guilty of it myself. > It's a bit like saying it's silly for pilots to have to know what to do in the case of engine failure or stall from memory and regularly practice in a simulator, because that stuff rarely happens and most of the time they're just using autopilot. That's true, it usually doesn't matter,…
I mean this is also what happens when pilots aren't following their checklists though.
Re: Green Lumber Fallacy in Software Engineering
#130I used have a similar perspective as the author, that the leetcode style questions / data structure / algorithm stuff they ask in interviews but seems to have little to do with day-to-day corporate software development is all a big waste of everyone's time. I'm not sure I still do, though. It's true that this stuff rarely comes up and often doesn't matter for most real-world applications... but when it does, it matte…
Over and over, I see code written by star professionals that you've heard of, reinventing the bubble sort. I'm guilty of it myself. > It's a bit like saying it's silly for pilots to have to know what to do in the case of engine failure or stall from memory and regularly practice in a simulator, because that stuff rarely happens and most of the time they're just using autopilot. That's true, it usually doesn't matter,…
if i need a toposort in my proprietary task scheduling system, or fisher–yates shuffle in my proprietary ranking system, what do you recommend? this isn't hypothetical - i've had these things come up in the last half at work.