Live data from Hacker News

The problems with live coding interviews

garrettdimon.com

51–60 of 226 posts

Re: The problems with live coding interviews

#51
post #5
post #4

Earlier quoted context omitted.

The last time I did a take home exercise for a position I wrote something that was beyond the team's skillset. That was awkward.

That's a great reason to do the take home test. You learn a huge amount about the team by how they react to your code.

Or they just evaluate among themselves and never tell you what they thought beyond thumbs up or thumbs down.

Re: The problems with live coding interviews

#52

Everyone complains about the state of SWE interviews yet nobody seems to be able to come up with a legit better alternative. Even the author recognises this in the article. This format is popular because it's the best time/effort trade-off for both the company and the candidate. It's massively flawed, but everything else attempted so far turned out to be even worse.

How do other industries do this? Is programming weird because you can just ask someone to prove they know how to use a hammer? And so other industries just have to hire based on work history and/or bias "culture fit" during the interview? And they suffer terribly from people who can talk the talk but not walk the walk? Or is programming weird because there's so much propensity for people to be able to talk but not wa…

I am not sure. I think part of the reason is that programming lives completely on the digital plane, while other fields tend to have physical components that can't be faked and allow you to grasp someone's knowledge in a much more straightforward way (e.g cooks produce physical food that you can see and taste)

Re: The problems with live coding interviews

#53

Your experience sounds horrible, and unfortunately those sort of "tests" are just a horrible thing. Generally, do NOT do take-home exercises. That's your time, which has value; the company doing the hiring contributes nothing. At least, in a live-coding exercise both you and they have some skin in the game. As with all tests, good or bad, part of the test is actually doing the test. Often it's less what you know, or…

That’s not always accurate that the company doing the hiring contributes nothing. (Also good companies could offer to pay for that take-home test time.) Having done take home before from the hiring side, it was incredibly time-consuming for us. We had someone anonymize the three finalist submissions, and then we had three people each individually review and comment on each one, and then we got together to discuss and…

There’s no guarantee the employer isn’t in final phases with someone else that will lead to them never even evaluating your work.

Re: The problems with live coding interviews

#54

Everyone complains about the state of SWE interviews yet nobody seems to be able to come up with a legit better alternative. Even the author recognises this in the article. This format is popular because it's the best time/effort trade-off for both the company and the candidate. It's massively flawed, but everything else attempted so far turned out to be even worse.

How do other industries do this? Is programming weird because you can just ask someone to prove they know how to use a hammer? And so other industries just have to hire based on work history and/or bias "culture fit" during the interview? And they suffer terribly from people who can talk the talk but not walk the walk? Or is programming weird because there's so much propensity for people to be able to talk but not wa…

Probably a little of both. A staggering number of candidates are just completely unable to solve simple problems.

Re: The problems with live coding interviews

#55

Earlier quoted context omitted.

There's also the issue of scalability. Grinding leetcode at least scales horizontally to a huge number of companies. Homework is typically useless outside of the single company you're doing it for. I generally refuse all takehome assignments unless: 1.) It sounds uniquely interesting and fun to do. 2.) The company is prestigious enough, or pays well enough, that making any effort to try to get the job worth it.

Completely agreed. Also in my experience the takehomes are even more susceptible to being rejected for absolute bullshit reasons. Someone told me the thing I coded up in less than an hour to solve their problem was “overengineered” so they wouldn’t be moving forward.

Also... I think at 100% of the companies that have given me a homework assignment, it has always been in addition to, not in place of, a standard leetcode whiteboard round.

Re: The problems with live coding interviews

#56
post #28

Live coding interviews are the future. The risk of false impersonation was low enough for a while to rely solely on virtual screens, but widespread access to and awareness of large language models has made any virtual exercise unsustainably corrupted. Sure, you can often tell the folks who know nothing when asking them to explain the code, but it's increasingly difficult to tell a great engineer apart from a mediocre…

One of the author's points is that the live coding questions are often the types of problems which, in the workplace, might best be done with GPT. "So at that point, do they want to see you muddle through it, or would they rather see that you know to have ChatGPT run through the initial pass and then refactor?" "If a company is evaluating engineers with questions that can be easily answered by AI in seconds, what are…

I think that’s not really relevant, in the same way that it doesn’t much matter when they take a football prospect’s running times that someone could go much faster on a bicycle or in a car.

Re: The problems with live coding interviews

#57
post #23
post #10

Earlier quoted context omitted.

> “take a novel problem that they haven't seen before and break it down” But who does that professionally as a stand-up performance, clock ticking, a judge breathing down your neck who has been equipped with a script that tells him things like “if candidate doesn’t do X within the first ten minutes it’s very bad”? Doing well in that situation depends more on social performance skills than problem-solving skills. You’…

As for the time limit, it's relative. All things equal , if you can complete the problem in 10 minutes and the company can hire somebody that can complete it in 9, then objectively the other candidate is the stronger one. As for nervousness for being forced to partaking in a stand-up performance, I'd argue that "social performance skills" can work against you, since the more "antisocial" you are, the more you can ign…

What are the "all things equal" here, though? I would tend to disagree that someone who can complete the coding challenge faster is the objectively better candidate. Especially if we are talking about leet coding, where there are people who study their asses off and memorize the common leet code problems. Or maybe the coding challenge isn't leet code, but just so happens to be something that the candidate has done before, or has done recently. Just because one candidate lucked out and has done this very specific thing, doesn't mean they are better than a candidate who could figure it out in half a day, if given a little more time to do so.

Re: The problems with live coding interviews

#58
These debates have been done to death and mostly I also think that the article is a fairly typical strawman argument. Take the worst way to do a live coding test, point out the problems and then dismiss the whole idea.

What else are we supposed to do? Take the fact that you can talk a good game as enough of a signal to invest 10s of 1000s? Assume that everyone with 20 years experience is as good as everyone else?

The problem is that there are no reliable signals. Most Developers I have interviewed have a massively inaccurate ability to judge their own ability (in both directions). I've lost count of the number of times candidates have rpomised that they can just learn whatever they don't already know and haven't been able to do it to any degree.

Qualifications are meaningful in some contexts more than others but most people in the UK don't have comp-sci qualifications.

So yes, I will use various coding exercises because depending on the level, it shouldn't phase someone to be given something quite simple and to see how they approach it (do they write tests first? Ask some good scope questions? Explain why they've done something the way they did?)

I have failed one of these tests in the past thinking I was a good Developer (I am!) but I don't blame the test or the process, I realised that my approach was haphazard and not an objective good look to an Interviewer so it was actually helpful.

Re: The problems with live coding interviews

#59
post #24

I’ve never tried this exactly, but I wonder how “live coding” interviews might go if the interviewer was the one doing the typing/coding and the candidate’s role was to verbalize what the interviewer was doing (and why) - and also guide them to some degree. It’d be more akin to pair-programming, which some of the interviews I’ve conducted have evolved into, depending on the strength of the candidates. Would that capt…

My first job split the programming interview in two. 45 minutes where the candidate drive (typed) and 45 minutes where the interviewer drove while the candidate guided them. The first half disqualified a lot more candidates than the second. It was usually pretty clear how the candidate would do by the end of the first half. Occasionally a candidate would be too limited in their communication to do well while not typing.

That job had a lot of people working in pairs very frequently so it made tons of sense to do it that way there.

Re: The problems with live coding interviews

#60
I think reviewing code is a much better method.

It's a real skill you'll actually employ. You're coming at the code cold, which is actually a realistic scenario you'll encounter on the job. Your ability to catch bad ideas and prevent them from getting literally codified is a valuable skill. And all of that is worthless if you're in a state where you can see a mistake, but are too afraid to speak up; this gets tested too.

It might not be so great for newbies and people fresh out of college, but even they should be able to read the code and discuss it.

Post reply on HN