Live data from Hacker News

The problems with live coding interviews

garrettdimon.com

161–170 of 226 posts

Re: The problems with live coding interviews

#161

Earlier quoted context omitted.

Maybe it's not insulting, but it sure as shit is pointless.

Based on my experience interviewing people I don't agree with that either.

Yeah there's plenty of people who would get badly hurt if they fell from the level they show in the CV (or the level of their ego) to the level of skills they can actually show.

Re: The problems with live coding interviews

#162

Your experience sounds horrible, and unfortunately those sort of "tests" are just a horrible thing. Generally, do NOT do take-home exercises. That's your time, which has value; the company doing the hiring contributes nothing. At least, in a live-coding exercise both you and they have some skin in the game. As with all tests, good or bad, part of the test is actually doing the test. Often it's less what you know, or…

I think we're going to see a pretty large industry shift away from take-home tests given how easily 90% of these tests can be gamed using chat GPT.

I'd think we'd have already seen this with GitHub Copilot. There was an interview I was part of late last year where the candidate had Copilot turned on during the live coding, and he didn't turn it off even after it was obvious that's what he was using. What I was more surprised by is how I thought this was a bigger deal than everyone else. Like, why come up with these elaborate tests when the candidate is just going to use autocomplete the whole time?

Maybe programming in another few years will just be glorified autocomplete and little more.

And perhaps testing people on how to write code was a mistake to begin with. It's one thing to write code, but reading code is another.

Re: The problems with live coding interviews

#163

Earlier quoted context omitted.

Your test sounds like what people complain about. You give them instructions but it's "really about ... ". Just ask them what you want them to know. If you hired this person and they turned in code without error recovery or encoding, you'd tell them they need to add it and they would. No need to obfuscate your intentions and have them read your mind in what you're looking for. No take home test is even needed. A simp…

> Just ask them what you want them to know. You can't really ask about those things directly. If I ask "(how) would you add monitoring to this?" you can make something up on the spot. If I ask "what else would a mature production app contain?" that will give me a better idea of what you're familiar with. > you'd tell them they need to add it and they would Different levels. We were not after juniors in that case, but…

When you say make it up on the spot, I'm hearing they can answer questions when you ask. If you ask them to describe how they'd process input and send it to a database they can just as easily tell you what they generally do. If they don't mention any of the things you're looking for, then you have your answer. If they answer some then you can decide to probe about what's missing, or thank them for their time. You haven't explained why a take home test is needed for that.

Re: The problems with live coding interviews

#164
post #89

It’s as simple as this: refuse any and all interviews that require you to dance I refuse to take those interviews. I had one company that promised they wouldn’t give me a live test, and when I told that to the interviewer who was trying to give me a test he said “hm, well we’re going to do it anyway” I passed the test and was given an offer which I shot down for the company which respected my terms In the end the oth…

Here's the problem. If the company offers you 600k salary then the dance is now worth it. The other problem is after lay offs, if all companies ask you to dance then you have no choice. Dance or stay unemployed. Obviously nobody wants to "dance" and nobody would if they had the choice.

So the only thing you can say in response is giving excuses why people must dance?

Sometimes you’re desperate and will do things you otherwise wouldn’t. That isn’t and should be thought of as the norm

Re: The problems with live coding interviews

#165

Recently I took a leetcode screening, my first ever, and failed to complete one single question in the allotted time due to a minor panic attack and a sudden loss of focus. The whole time I couldn't help but think about the state of the economy and how "this is it, this is my only chance." Half the time I was fighting with the in-browser IDE and struggling to remember names of JavaScript builtins that usually come ea…

I always bomb the whiteboard coding portion. I've been doing this for over 20 years. I've been a senior engineer since before the iPhone existed.

Fortunately, I interview well and can discuss many, many technical concepts deeply. But put me in front of a whiteboard and I couldn't tell you my mom's name.

Re: The problems with live coding interviews

#166
post #118
post #42

Earlier quoted context omitted.

Normal interviews are mostly a scripted performance where people largely regurgitate answers to predictable questions. So the issue isn’t that it’s a stand up performance it’s that you need to split your attention between the performance and solving some trivial problem. Becoming really good at programming requires being able to focus on an actually difficult problem not work through a simplified example while entert…

> Normal interviews are mostly a scripted performance where people largely regurgitate answers to predictable questions. Ah, so you prefer being asked "where do you see yourself in 5 years?", and regurgitating the text generated by ChatGPT instead? Alternatively, as I replied in the sibling thread, do you prefer to be handed a difficult take home task that is expected to take 2 weeks to complete? I think I can live w…

“Where do you see yourself in 5 years?” is asking a perfectly reasonable question about what if anything you’re working towards. If you think regurgitating something from ChatGPT is appropriate, I can see why you might prefer more limited interviews.

Re: The problems with live coding interviews

#167

The problem with live coding interviews is that they shouldn't exist. Another problem is that we're prone to thinking that being able to do well on tests equates to doing well in life and work--despite a stunning lack of evidence in support it.

So what do you propose? Picking names out of a hat?

[deleted]

Re: The problems with live coding interviews

#168

Earlier quoted context omitted.

> Just ask them what you want them to know. You can't really ask about those things directly. If I ask "(how) would you add monitoring to this?" you can make something up on the spot. If I ask "what else would a mature production app contain?" that will give me a better idea of what you're familiar with. > you'd tell them they need to add it and they would Different levels. We were not after juniors in that case, but…

When you say make it up on the spot, I'm hearing they can answer questions when you ask. If you ask them to describe how they'd process input and send it to a database they can just as easily tell you what they generally do. If they don't mention any of the things you're looking for, then you have your answer. If they answer some then you can decide to probe about what's missing, or thank them for their time. You hav…

> When you say make it up on the spot, I'm hearing they can answer questions when you ask.

Or they have enough surface knowledge to risk an answer that could potentially work. It depends if that's the threshold acceptable to you.

> You haven't explained why a take home test is needed for that.

We're commenting on the blog post about this very question. Making the initial stage of this test take-home is at least partially solving points 2,5,6,8 from the list.

Re: The problems with live coding interviews

#169
After having been through this crap and now being on the other side of the desk, I've come to the conclusion that this is simply standardized testing for adults, with all of the same myriad problems. It doesn't identify what you're actually looking for, gives you more false signals, and alienates the talent you actually want.

The way we do these at my current job is extremely productive for us. We look for two things, apart from basic competency: problem solving, and asking for help.

It's structured as a two 90 minute pair programming sessions. Interviewee shares their screen, and we work through the problems together. Obviously it's pretty hands-off, but we guide and nudge where appropriate. Here and there, when they use something that relates to a deeper topic, I'll ask questions to gauge how deep their knowledge goes. Like asking if they know how C#'s foreach works under the hood. Not as a selection criteria, but simply to get a sense of how much they know.

Use of a search engine is openly encouraged. A lot of the time, we don't even care if the program actually runs. If they struggle with syntax or the correct function overload, we'll help them out after giving them a little time to find the solution.

I also throw in a problem designed to get them stuck, and ask questions I expect they can't answer. A good programmer asks for help and admits when they don't know. A bad one bullshits their way through.

We want to hire programmers who can do a real job in the real world. Implementing red/black trees on a whiteboard blindfolded isn't a job skill, it's a party trick. That's not something a programmer will ever need to do.

In the real world, real people use google and stack overflow. They don't have encyclopedic knowledge of the entire language's syntax. They ask their coworkers for help or opinions.

Our interview process is designed to show us how a person will function in a scenario as close to the job as possible. Because that's what we're hiring them for. We look for their ability to work through a problem with the resources that everyone always has. We look for how they work with others and how much they lean on coworkers.

This has worked out extremely well for us. We've hired some very talented individuals, and have totally avoided the archetypal shitty dev. The people we hire immediately mesh with the team, and learn and grow the way all programmers do.

Granted, we are a small company and we have the time to have our own programmers giving interviews. We also have a much higher need to be so selective. But every single person who has made it to the technical interview has remarked unprompted that it's the best interview they've ever had. And I mean 100%.

It's because we treat candidates the way we'd treat our own employees. They get to know what the job is like, and we get to know how they'll do the job.

Re: The problems with live coding interviews

#170

I have been in the software industry for 30 years as of this month. I have never had a gap of longer then 6 months and the shortest time with a company was 4 years 8 months and the longest time was 16 years. I have worked on the following technoclogies. Companies range from on of the Big 3 consulting firms to startups. Here are the technologies I have worked on - C/C++ on Win16/Win32 - Assembly language development w…

> Relational DB (MsSQL, PostGres, MySql), NoSQL db (MonoDB)

> At my experience can we just assume that I am a competent coder and talk about the job and how I can contribute ? I mean its almost insulting if you ask me to make a linked list/reverse a binary tree or other such nonsense looking over my shoulder me with a time limit.

I might inclined to agree had you not gotten all the names of the databases in your list wrong.

Post reply on HN