Live data from Hacker News

The problems with live coding interviews

garrettdimon.com

91–100 of 226 posts

Re: The problems with live coding interviews

#91
post #79

Earlier quoted context omitted.

> Detecting a cycle in a directed graph is useful (in certain domains I've worked in), but doesn't have a cute trick answer. Doesn't the standard cute trick answer for a linked list (tortoise and hare) work fine? Do two parallel breadth-first traversals of the graph starting at the root node, one at twice the speed of the other. (If there's multiple root nodes, introduce a unique super-root that points to the roots.)…

if you can mutate the graph and mark a node as traversed that's the easiest way. If you can't then save the addresses to a hash set. Both are extra memory but you don't have to deal with "parallel BFS," which honestly is just over complicated imo. With "parallel bfs" comparing "layers" of traversal has runtime cost that effects the big oh so I think the above solution is better overall even though it feels cheap.

But the whole point of the "cute solution" is to solve with O(1) memory; if you're willing to allow O(n) then the linked list also admits more sane solutions.

I'm not sure what you mean by comparing "layers" of traversal. Tortoise and hare is about provide a termination condition in the case that a cycle exists. In the case that there's no cycle, every algorithm must inspect each node, so is O(n). In both the linked list and the DAG case, if there is at least one cycle, (a) both pointers will enter it in at most O(n) time, and they will match in at most O(n) additional time, so this stays big-oh optimal.

Re: The problems with live coding interviews

#92

Earlier quoted context omitted.

How do other industries do this? Is programming weird because you can just ask someone to prove they know how to use a hammer? And so other industries just have to hire based on work history and/or bias "culture fit" during the interview? And they suffer terribly from people who can talk the talk but not walk the walk? Or is programming weird because there's so much propensity for people to be able to talk but not wa…

Probably a little of both. A staggering number of candidates are just completely unable to solve simple problems.

Many years ago, I interviewed a candidate at Amazon for an SDE3 job (aka Senior SDE.) I asked him to write a function that takes 2 sorted arrays of integers and merges them into a single sorted array of integers. He just couldn't do it.

This is very simple problem to solve, and I think anyone who can't do that is not suitable for SDE3 (or 2 or 1, in my opinion)

One could argue about very difficult problems - can a candidate really be expected to invent the solution on the spot, or is it just because they saw it on LeetCode? Is it luck or ability?

But for simple problems, they are a strong negative signal and a very weak positive signal: just because you can merge sorted arrays, doesn't mean you're qualified, but if you can't even do that, you're most likely not qualified.

Re: The problems with live coding interviews

#93
post #28

Earlier quoted context omitted.

One of the author's points is that the live coding questions are often the types of problems which, in the workplace, might best be done with GPT. "So at that point, do they want to see you muddle through it, or would they rather see that you know to have ChatGPT run through the initial pass and then refactor?" "If a company is evaluating engineers with questions that can be easily answered by AI in seconds, what are…

I think that’s not really relevant, in the same way that it doesn’t much matter when they take a football prospect’s running times that someone could go much faster on a bicycle or in a car.

I'm pretty sure I heard the same point made 20 years ago against using IDEs, with their auto-complete and tool-tips and style feedback.

And 40 years ago in the debate over letting kids use a calculator in school instead of calculating by hand.

A footballer follows a very constrained set of rules. If the footballer were allowed to use a car then 1) it would be easy to score a point, 2) the field would be ruined, and 3) people wouldn't pay to watch or support the team.

If a programmer uses ChatGPT to get a handle on a task with a new API, and saves a day of futzing around, how is that NOT relevant to job performance? (For the sake of argument, let's say that experiment showed that API doesn't scale well enough, resulting in a decision to scrap that approach entirely and use a different API.)

Re: The problems with live coding interviews

#94
post #89

It’s as simple as this: refuse any and all interviews that require you to dance I refuse to take those interviews. I had one company that promised they wouldn’t give me a live test, and when I told that to the interviewer who was trying to give me a test he said “hm, well we’re going to do it anyway” I passed the test and was given an offer which I shot down for the company which respected my terms In the end the oth…

Here's the problem. If the company offers you 600k salary then the dance is now worth it.

The other problem is after lay offs, if all companies ask you to dance then you have no choice. Dance or stay unemployed.

Obviously nobody wants to "dance" and nobody would if they had the choice.

Re: The problems with live coding interviews

#95
I feel all discussions about the coding interview are missing two critical points: first, how we got here, and second, why are we still here.

First, how we got here. I didn't experience this first hand, so I'm just making a recount based on what I've heard and read from "old timers". My understanding is that before whiteboard, leetcode-style interviews became the norm, tech interviews were mostly unstructured and quite informal. In the really old times (pre 90s) you could get a job just by knowing how to use a computer. I believe this wasn't that unusual in the 90s and early 2000s. I still remember hearing a founder-CEO bragging that his interview process was a 30 min chat with each candidate, and if he liked them, he would hire them.

From what I could gather Microsoft is the first big company that started using these "coding challenges". Then they became wildly popular thanks to Joel Spolsky [1], the publishing of Cracking the Coding Interview [2], and Google made brain teasers world famous.

Second, why we are still here. First, I believe there is a huge cargo-cult factor. Companies want to copy big tech, and alumni from these companies go on to found their own. This kind of interview has been honed and polished over the years, landing in a local optimum. An entire industry of websites and products has been created, and there are many entrenched interests. People might hate it, but the process works well enough for tech companies that they don't need to worry too much about it. Another under-appreciated factor is scale. This kind of interview sort-of scales well, which is important when you hire at a massive scale. That's why things like "have the candidate come to work one day and pay them" won't work for companies that need to screen thousands of people a week. Lastly, a standardized and "well known" process introduces some guardrails that can avoid some obvious pitfalls. The book "Working Backwards" explain how the Bar Raiser program was created at Amazon when a bad senior leader hire used the unstructured approach to hiring to build an empire misaligned with the company. At a big enough company this is bound to happen sooner or later.

Third, where do we go from here? I find it extremely unlike existing big companies will change their methodology any time soon. It might not seem like it, but for a company like Google it would be a massive undertaking to overhaul their hiring process. It would take years to achieve the level of efficiency and effectiveness of the current system, and surely there would be tons of opposition.

I believe the only way forward is for new companies to experiment with other methods of hiring, particularly at the beginning when they are nimble and can experiment freely. As they grow they will face challenges scaling, polishing and standardizing their process. At some point they will become the next generation of Big Tech, and the cargo cult wheel might spin again.

In any event it seems we need some sort of structured approach to hiring where we assess the match between company and candidate.

[1] https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid...

[2] https://www.crackingthecodinginterview.com/

Re: The problems with live coding interviews

#96
post #79
post #61

Earlier quoted context omitted.

Detecting a cycle in a directed graph is useful (in certain domains I've worked in), but doesn't have a cute trick answer. [Edit: I in mind ‘Detecting the cycles’ rather than ‘Detecting a cycle’ but wrote the wrong thing. Mea culpa.] Your actual job will be more like implementing a maximally performant directed graph in safe Rust. (Not really. It won't be that interesting.)

> Detecting a cycle in a directed graph is useful (in certain domains I've worked in), but doesn't have a cute trick answer. Doesn't the standard cute trick answer for a linked list (tortoise and hare) work fine? Do two parallel breadth-first traversals of the graph starting at the root node, one at twice the speed of the other. (If there's multiple root nodes, introduce a unique super-root that points to the roots.)…

Yes, I phrased my comment incorrectly; I had in mind identifying (all) the cycles, and that for a linked list they're the same thing. For a DAG it doesn't generalize because you can't find more cycles without storage to exclude the earlier ones.

Re: The problems with live coding interviews

#97
post #93

Earlier quoted context omitted.

I think that’s not really relevant, in the same way that it doesn’t much matter when they take a football prospect’s running times that someone could go much faster on a bicycle or in a car.

I'm pretty sure I heard the same point made 20 years ago against using IDEs, with their auto-complete and tool-tips and style feedback. And 40 years ago in the debate over letting kids use a calculator in school instead of calculating by hand. A footballer follows a very constrained set of rules. If the footballer were allowed to use a car then 1) it would be easy to score a point, 2) the field would be ruined, and 3…

We’re not talking about whether you should use them at work; we’re talking about whether it makes sense to have an evaluation where you don’t use them. Closed-book exams are similar. There’s no real-world situation where you wouldn’t be allowed to refer to whatever materials you like, but the evaluation uses somewhat unnatural circumstances to gauge how well you’ve assimilated the material in a limited amount of time and with a consistent process that’s fair for everyone.

Re: The problems with live coding interviews

#98
post #96
post #79

Earlier quoted context omitted.

> Detecting a cycle in a directed graph is useful (in certain domains I've worked in), but doesn't have a cute trick answer. Doesn't the standard cute trick answer for a linked list (tortoise and hare) work fine? Do two parallel breadth-first traversals of the graph starting at the root node, one at twice the speed of the other. (If there's multiple root nodes, introduce a unique super-root that points to the roots.)…

Yes, I phrased my comment incorrectly; I had in mind identifying (all) the cycles, and that for a linked list they're the same thing. For a DAG it doesn't generalize because you can't find more cycles without storage to exclude the earlier ones.

Fair enough. I imagine a simple (but O(cycles) space, not O(1)) way to do this is to still tortoise-and-hare, and on detecting a cycle, break the link behind you (by recording it in a table) and continue the BFS. If O(cycles) = O(n) this isn't terribly interesting compared to other approaches, but if O(cycles) < O(n) it might be helpful.

Re: The problems with live coding interviews

#99
post #98
post #96

Earlier quoted context omitted.

Yes, I phrased my comment incorrectly; I had in mind identifying (all) the cycles, and that for a linked list they're the same thing. For a DAG it doesn't generalize because you can't find more cycles without storage to exclude the earlier ones.

Fair enough. I imagine a simple (but O(cycles) space, not O(1)) way to do this is to still tortoise-and-hare, and on detecting a cycle, break the link behind you (by recording it in a table) and continue the BFS. If O(cycles) = O(n) this isn't terribly interesting compared to other approaches, but if O(cycles) < O(n) it might be helpful.

Actually, now that I spend more than ten seconds thinking about this, I think this requires O(1) space; you only need to remember the single most recent link behind you to avoid following it again. Remembering a single break is sufficient to get you to either the next cycle or (if no more) terminate the BFS; and once you detect the next cycle you can safely remember only /its/ break since BFS will never take you back to the first cycle. Assuming streaming output (e.g. print identification of each cycle as you detect it -- including e.g. printing each node until you see the break point, which will identify each node that is part of the cycle), this is O(n) time and O(1) space, which I believe is optimal.

Re: The problems with live coding interviews

#100

I think reviewing code is a much better method. It's a real skill you'll actually employ. You're coming at the code cold, which is actually a realistic scenario you'll encounter on the job. Your ability to catch bad ideas and prevent them from getting literally codified is a valuable skill. And all of that is worthless if you're in a state where you can see a mistake, but are too afraid to speak up; this gets tested…

I agree, and this is what we have been doing for a while at my current job (I went through the same process, so it's been done for at least 7 years).

The candidate would come to the office (when we were still doing in-person interviews) and meet a couple of the developers for a little bit of chatting. They would then be given a couple of printed pages of Java code with a few basic classes representing banking accounts with some typical methods for depositing, withdrawing, persisting to a database etc. (with all of the details stubbed out) - and they would be given these instructions:

  Read the provided Java source code files and identify any problems that you can find.
  The problems can include things such as actual bugs, design problems or code quality issues.
  You do not need to search for syntax errors - the code does compile.
  You have 15 minutes to find as many problems as you can.
 
The candidate would be left alone with the papers and a pen and would spend the next 15 minutes looking over the code by themselves.

The rest of the interview would then be spent discussing their findings. Most candidates would find the obvious problems in the logic, missing null-checks etc., while trickier things like synchronization issues were missed by quite a few. Even though we had a list of all the bugs/issues that had been put into the code, the important part wasn't for a candidate to check off as many of these as possible - the important part was the discussion about the issues that followed.

After the candidate had told us what they found, we would start hinting about the remaining issues and eventually tell about all of them. How quickly someone would pick up on an issue when it was pointed out told us quite a lot. It was a way to get a feel for how the candidate thought and reasoned about code, without the pressure of them having to actually write code with someone looking over the shoulder.

Post reply on HN