Green Lumber Fallacy in Software Engineering • Hacker News Board

Re: Green Lumber Fallacy in Software Engineering

#151

Post by eropple » Fri, Jan 14, 2022, 5:23 AM UTC

tacostakohashi wrote: post #115

I used have a similar perspective as the author, that the leetcode style questions / data structure / algorithm stuff they ask in interviews but seems to have little to do with day-to-day corporate software development is all a big waste of everyone's time. I'm not sure I still do, though. It's true that this stuff rarely comes up and often doesn't matter for most real-world applications... but when it does, it matte…

> It's true that this stuff rarely comes up and often doesn't matter for most real-world applications... but when it does, it matters a lot. The difference between O(n2), O(n) and O(log n) is huge for big values of n. Being about to go straight to a (nearly) optimal solution, instead of hacking together something, waiting for it to burst a the seams, and then having to fix it... or being able to spot someone else doing that, can save weeks or months of development at a time.

This is true, and absolutely matters. But I would suggest that developing the intuition to recognize a problem as being of a particular class, and even going further as to being able to look at the provided solution and intuit whether or not it is optimal (or optimal-enough), is a different thing than calling up that optimal algorithm from your limited brainspace.

Like--if you understand the principles of what you're doing, Google isn't a much slower recall mechanism than you are, and it's less likely to have misremembered something.

Re: Green Lumber Fallacy in Software Engineering

#152

Post by eropple » Fri, Jan 14, 2022, 5:30 AM UTC

xsmasher wrote: post #113

lhorie wrote: post #104

Earlier quoted context omitted.

One needs to be especially careful with adapting experience to interview questions. Unlike the candidate, you probably were already immersed in the problem domain for months or years and have good context in what techniques exist and are relevant to the problem domain. One mistake some interviewers make is implicitly assuming that candidates can somehow conjure the same level of context from first principles, or that…

Every time I feel "this would be a good interview question!" it is not. It's usually something juicy that I chewed on for two days. How can I expect a candidate to solve it in 45 minutes?

I've had that same feeling on that side of the table, too. Like some of the stuff I've run into is really cool and really rewarding and is really tempting because it would make an interview something more fun for me. Because I know the answer.

Over time I've learned that I'd kind of rather lean a little more towards the easier side than the harder for writing code during an interview, because the interview is unpredictably stressful. But at the same time, as prioritizing communication and a degree of thoughtfulness has become more important (which has ended up with me bopping over to a devex job where I am now), I've leaned more heavily on "let's talk through XYZ and suss out how you discursively approach the problem" types of interviews. Which definitely selects for a particular audience, but it's one more useful for the roles I've hired for.

Re: Green Lumber Fallacy in Software Engineering

#153

Post by random314 » Fri, Jan 14, 2022, 5:30 AM UTC

YZF wrote: post #118

tacostakohashi wrote: post #115

I used have a similar perspective as the author, that the leetcode style questions / data structure / algorithm stuff they ask in interviews but seems to have little to do with day-to-day corporate software development is all a big waste of everyone's time. I'm not sure I still do, though. It's true that this stuff rarely comes up and often doesn't matter for most real-world applications... but when it does, it matte…

When it comes up do you need to code it up in 20 minutes or is looking the algorithm up in a textbook ok? If all you want to test that sort of knowledge just ask the knowledge question. I don't need to memorize Floyd-Warshal and be able to code it on demand to be able to recognize when it's needed and look it up. I'm sure I remembered it back in school, haven't used it since, but if the day comes I'll look it up...

I have worked with engineers who treated their work as glueing together various services to get the results required. They were absolutely shit engineers, whose approach to building services for massive scale were simply

1. Glue together whatever services are available and seem to make sense.

2. Throw increasing amounts of traffic at it and check what fails

3. Try to patch around failure point

4. blame the service provider for "not scaling"

5. Ask for more and more machines

6. Exclaim how great their design is on the basis of the cool class hierarchies and interfaces they wrote. Teach "junior devs" their "OOP design skills and patterns"

7. Go to step 2

They think of themselves as great engineers who scale things. The reality is they are absolute shit engineers who have no clue what they are building, frequently have crazy O(n^2) loops in their code, dont recognize a graph problem if its staring them in their face.

Examples

1. They needed a way to rerun map reduce jobs to regenerate data for previous 30 days. It was a straight forward, graph topological sort problem. Instead they built a crazy solution using multiple instances of Airbnb airflow. One server ran out of memory for these airflow instances, so they allocated a whole damn cluster of machines to run 30 instances of airflow! The job of the cluster? Issuing map reduce jobs to the hadoop cluster. They needed a cluster of machines just to issue commands to hadoop. They gave an elaborate tech talk about "scaling to 30X capacity" to process 30 days of data. Meanwhile, my team was processing 3 years of data. We considered hamake for our data rebuild process. Eventually, I wrote a 70 line scala topological sort routine that would do the job for us, because it gave us more flexibility than hamake and linked to our code libraries.

2. Some guy wrote a data pipeline to summarize a days results that took 24 hours to run, thanks to 40 different joins. I rewrote it to use a single group by. The code ran in 10s of minutes. He didn't know how hadoop worked or external sorting, beyond writing arbitrary sql. So much for looking up algorithms when required.

3. Guys presents beautiful OOP code that has major race conditions in a distributed environment. I point out the race condition. Keeps proposing hacks to "solve the problem". Does not understand transaction processing or locking algorithms or what options are available to him. Hadn't heard if read write locks. What are the pros or cons. Doesn't understand the difference between local locks and distributed fault tolerant locks. I explain it to him. He eventually comes up with a solution using zookeeper that will need zookeeper to process 1000s of updates per second. I give up.

"DSA, what DSA? They are for kids. I will look up an algorithm when I need it".

Re: Green Lumber Fallacy in Software Engineering

#154

Post by tokipin » Fri, Jan 14, 2022, 5:33 AM UTC

tubby12345 wrote: post #144

YZF wrote: post #141

Earlier quoted context omitted.

I can assure you, 100% without a doubt, that this is where the vast majority of developers are. Including people working on the most cutting edge software. The only reason the people you interview(?) can regurgitate this is because they just re-read their course book yesterday and they expect algorithmic problems. Or sure, some of us do competitive coding and such where these things do show up. There's absolutely no…

>I can assure you, 100% without a doubt, that this is where the vast majority of developers are. cool? then that majority doesn't generate the same value as those that do and so they shouldn't get paid as such? i already asserted that? >Including people working on the most cutting edge software. ctrl+f to doubt that you know what people working on cutting edge software do. i, on the other hand, do have some experienc…

> they're prepared for tasks being assigned to them that exercise those skills and by the end of the half they'll have a deep understanding rather than just a wrote understanding.

I think it's entirely possible to memorize algorithms superficially. These types of interviews are selecting for exactly the type of person that is good at that rote memorization, and are otherwise selecting for a very narrow type of skillset.

It's unreasonable to not expect pathological outcomes with such a rigid system, especially when some people are themselves specialized for the ability to game social systems. Nothing is free. There are always trade offs when choosing one approach over another.

Re: Green Lumber Fallacy in Software Engineering

#155

Post by TulliusCicero » Fri, Jan 14, 2022, 5:34 AM UTC

vanusa wrote: post #148

compiler-guy wrote: post #7

The test isn't so much about the ability to invert a binary tree (for example), but the ability to problem solve complicated problems where the obvious solution is probably not the best one. In interviewing, it is hard to produce real world problems that don't require a huge amount of context and details of some obscure software stack, so we go with abstract problems which are kind of irrelevant to real work, but the…

But the ability to problem solve complicated problems where the obvious solution is probably not the best one. Problems like designing "the most popular package manager on macOS", for example? I think what the author finds strange is that not only do many people still believe that LC (or whiteboarding) are effective proxies for general problem solving ability -- they seem to tacitly believe they are better measures t…

> Problems like designing "the most popular package manager on macOS", for example?

Yes, those fall through the gaps, but people with that kind of extremely strong, publicly viewable history are much more the exception than the norm.

Re: Green Lumber Fallacy in Software Engineering

#156

Post by eropple » Fri, Jan 14, 2022, 5:36 AM UTC

amscanne wrote: post #122

dragonwriter wrote: post #114

Earlier quoted context omitted.

> If you apply a standardized test, then it becomes feasible to measure and argue the disparate impact. Yes, the existence of readily available statistics may make the unequal impact easier to show. > For all their failings, leetcode-style interviews are probably less culturally-sensitive than standardized IQ tests. “Culturally sensitive”, maybe, but they almost certainly have quite large unequal impacts adverse to p…

You are committing a corollary to the green lumber fallacy described by the article. Just because code challenges are not relevant to job duties doesn’t mean the results are irrelevant to job performance . They are a proxy intelligence test. General intelligence is the best predictor of success in almost any role (not just software engineering). This is what confounds people. They think the interviews are designed th…

> They think the interviews are designed that way because they are supposed to be representative of the job — I don’t believe that’s the case.

This may be the case. But even if it does act as an effective proxy--and I'm not sure it does; my worst yeses in interview loops have been very adroit programmers who I passed against my better judgment when my "not sure I want to be around this person every day" bells were ringing subtly in the back of my mind--then it has a different problem. You've now set expectations with the interviewee that oh yeah, we do hard stuff here. Then they go frob knobs or write frontend stuff all day.

(This actually happened to me at my first job. I didn't know any better, of course. The hiring manager pumped up my tires with all the difficult scaling work, etcetera etcetera. Then I was writing HTML into templates for the first six months I was there because they needed a body to plug into the role.)

Re: Green Lumber Fallacy in Software Engineering

#157

Post by vertr » Fri, Jan 14, 2022, 5:40 AM UTC

MarkMarine wrote: post #54

mrkentutbabi wrote: post #42

If I have money for every article that rail against DS&A algorithm.... I don't understand why people hate DS&A so much. Just do it. Get money, jump ship, get more money, it also makes you a better engineer. It gives you better compensation, it helps you, helps everyone, helps your team, your company. It helps avoid wasting time as well on both sides. There is no downside of studying DS&A and Leetcode, if there is, th…

I came here to basically say this. You want the big salary at the fancy company, show me you put in the work to earn it.

No post body was provided.

Re: Green Lumber Fallacy in Software Engineering

#158

Post by tubby12345 » Fri, Jan 14, 2022, 5:41 AM UTC

tokipin wrote: post #154

tubby12345 wrote: post #144

Earlier quoted context omitted.

>I can assure you, 100% without a doubt, that this is where the vast majority of developers are. cool? then that majority doesn't generate the same value as those that do and so they shouldn't get paid as such? i already asserted that? >Including people working on the most cutting edge software. ctrl+f to doubt that you know what people working on cutting edge software do. i, on the other hand, do have some experienc…

> they're prepared for tasks being assigned to them that exercise those skills and by the end of the half they'll have a deep understanding rather than just a wrote understanding. I think it's entirely possible to memorize algorithms superficially. These types of interviews are selecting for exactly the type of person that is good at that rote memorization, and are otherwise selecting for a very narrow type of skills…

No post body was provided.

Re: Green Lumber Fallacy in Software Engineering

#159

Post by vanusa » Fri, Jan 14, 2022, 6:01 AM UTC

TulliusCicero wrote: post #155

vanusa wrote: post #148

Earlier quoted context omitted.

But the ability to problem solve complicated problems where the obvious solution is probably not the best one. Problems like designing "the most popular package manager on macOS", for example? I think what the author finds strange is that not only do many people still believe that LC (or whiteboarding) are effective proxies for general problem solving ability -- they seem to tacitly believe they are better measures t…

> Problems like designing "the most popular package manager on macOS", for example? Yes, those fall through the gaps, but people with that kind of extremely strong, publicly viewable history are much more the exception than the norm.

Does have to be not just "strong" -- but "extremely strong" evidence?

Not to split hairs - but it does seem to be a peculiar choice of words, there.

I just don't see why the DSA filter has come to be considered so golden that one's ability to not just demonstrate some baseline capacity for this skillset, but to positively master a sequence of concocted performative rituals around it (specifically: reciting these algorithms in front of groups of strangers at a whiteboard; "talking" large blocks of code to someone over a phone; or doing online tests under completely unrealistic time pressure) -- that, in some corners at least, has come to categorically dominate all other selection criteria in the hiring process.

(Actually I do see why -- but it all comes down to factors not related to the intrinsic importance of these skills, or to whether the candidate can effectively do the job).

Re: Green Lumber Fallacy in Software Engineering

#160

Post by lordnacho » Fri, Jan 14, 2022, 6:09 AM UTC

jcrites wrote: post #103

lhorie wrote: post #89

Earlier quoted context omitted.

Yeah, that is one of those claims that sound nice in theory but kinda come crashing down when rubber meets the road. Companies do in fact engage in a process of getting someone in and doing an actual job, they're called internships. The problem is that these arrangements typically involve someone dedicating some non-trivial amount of their time to "babysitting" (not necessarily in the literal sense, but in the sense…

> Literally setting up a paid one week period for a senior level candidate would cost, optimistically, a few hundred dollars for evaluating a single candidate. It’s unworkable for anyone who is already employed. What am I going to do: use a week of vacation for this “interview”? (The fact that I’d be paid is immaterial: I lose vacation time. Plus, the more senior you are, the more of your compensation is in equity. W…

This might change when a lot of people are remote. I could see people working a second job temporarily to see if they want to jump ship.