Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI killed the tech interview. Now what?

kanenarraway.com

Post reply on HN
Print view View original

211–220 of 651 posts

Re: AI killed the tech interview. Now what?

#211

Post by rachofsunshine »

mattkrause wrote: post #179
rachofsunshine wrote: post #166

The death of shitty interviews has been greatly exaggerated. AI might make e.g. your leetcode interview less predictive than it previously would have been. But was it predictive in the first place? I don't think most interviews are written by people thinking in those terms at all. If your method of interviewing never depended on data suggesting it actually, you know, worked in the first place, why would it matter if…

> A lot of these people … just suck. Another possibility is that their job subtly drifted. I wrote a lot of code as a grad student but my first interviews afterward were disasters. Why? Because I’d spent the last few months writing my thesis and the few months before that writing a very specific kinds of code (signal processing, visualization) that were miles away from generic interview questions like “Make the longe…

We don't ask "make the longest palindrome". We ask "convert this English into code that does what it says". If you want to make the discussion more concrete, we have a public practice problem [1] that we send out with our interview bookings so that people know what to expect. The real problems we ask are very similar to it.

Do you feel like there's anything there that any reasonably skilled programmer shouldn't be able to figure out on the fly?

[1] https://www.otherbranch.com/shared/practice-coding-problem

Re: AI killed the tech interview. Now what?

#212

Post by CharlieDigital »

conjectures wrote: post #192
jghn wrote: post #139

Earlier quoted context omitted.

This is harder than it sounds, although I agree in a vacuum the idea is a good one. So much value of the code review comes from having actual knowledge of the larger context. Mundane stuff like formatting quirks and obvious bad practices should be getting hoovered up by the linters anyways. But what someone new may *not* know is that this cruft is actually important for some arcane reason. Or that it's important that…

It's not so hard. One of the interview stages I did somewhere well known used this. Here's the neural net model your colleague sent you. They say it's meant to do ABC, but they found limitation XYZ. What is going on? What changes would you suggest and why? Was actually a decent combined knowledge + code question.

There are so many interesting ways to use code reviews like subtly introducing defects and bugs and see if people can follow the logic, read the code, find where the reasoning comes up short.

I wrote up 7 general strategies for teams that are interested: https://coderev.app/blog/7-strategies-for-using-code-reviews...

Re: AI killed the tech interview. Now what?

#213

Post by apwell23 »

ghaff wrote: post #176
tempodox wrote: post #167

Earlier quoted context omitted.

Personal networks only disadvantage those who have none.

I suspect many people who don't have strong networks for whatever reason resent that. To which you could probably tack on not having gone to the "right" schools or having a public portfolio.

also hard on introverts who already get punished in workplaces that promote ppl based on proximity and visiblity.

Re: AI killed the tech interview. Now what?

#214

Post by jakubmazanec »

datadrivenangel wrote: post #196
jakubmazanec wrote: post #67

The problem isn't AI, the problem is companies don't know how to properly select between candidates, and they don't apply even the basics of Psychometrics. Do they do item analysis of their custom coding tests? Do they analyse the new hires' performances and relate them to their interview scores? I seriously doubt it. Also, the best (albeit the most expensive) selection process is simply letting the new person to do…

How do you control for confounders and small data? For data size, if you're a medium-ish company, you may only hire a few engineers a year (1000 person company, 5% SWE staff, 20% turnover annually = 10 new engineers hired per year), so the numbers will be small and a correlation will be potentially weak/noisy. For confounders, a bad manager or atypical context may cause a great engineer to 'perform' poorly and leave…

Sure, psychological research is hard because of this, but that's not what I'm proposing - I'm talking about just having some data on predictive validity of the hiring process. If there's some coding test: is it reliable and valid? Aren't some items redundant because they're too easy or too hard? Which items have the best discrimination parameter? How the total scores correlate with e.g. length of the test takers tenures?

Sure, the confidence intervals will be wide, but it doesn't matter, even noisy data are better than no data.

Maybe some companies already do this, but I didn't see it (though my sample is small).

Re: AI killed the tech interview. Now what?

#215

Post by advael »

I think that a mythology about where the difficulty in working with computers lies has made the relationship between businesses and the people they hire to do this stuff miserable for quite some time

"Coding", as in writing stuff in programming languages with correct syntax that does the thing asked for in isolation, has always been a very dumb skill to test for. Even before we had stackoverflow syntactic issues were something you could get through by consulting a reference book or doing some trial and error with a repl or a compiler. That this is faster now with internet search and LLMs is good for everyone involved, but the fact that it's not what matters remains

The important part of every job that gets a computer to do a thing is a combination of two capabilities: Problem-solving, that is, understanding the intended outcome and having intuition about how to get there through whatever tools are available, and frustration tolerance: The ability to keep trying new* stuff until you get there

Businesses can then optimize for things like efficiency or working well with others once those constraints are met, but without those capabilities you simply can't do the job, so they're paramount. The problem with most dinky little coding interviews wasn't that you could "cheat", it's thst they basically never tested for those constraints by design, though some clever hiring people manage to tweak them to do so on an ad hoc basis sometimes

* important because a common frustration failure mode is repetitive behavior. Try something. Don't understand why it doesn't work. Get more frustrated. Try the same thing again. Repeat

Re: AI killed the tech interview. Now what?

#217

Post by michaelt »

perlgeek wrote: post #132

Company A wants to hire an engineer, an AI could solve all their tech interview questions, so why not hire that AI instead? There's very likely a real answer to that question, and that answer should shape the way that engineer should be assessed and hired. For example, it could be that the company wants the engineer to do some kind of assessment whether a feature should be implemented at all, and if yes, in what way.…

> Company A wants to hire an engineer, an AI could solve all their tech interview questions, so why not hire that AI instead?

Interview coding questions aren't like the day-to-day job, because of the nature of an interview.

In an hour-long interview, I have to be able to state the problem in a way the candidate can understand, within 10 minutes or so. We don't have time for a lecture on the intricacies of voucher calculation and global sales tax law.

It also has to be a problem that's solvable within about 40 minutes.

The problem needs to test the candidate meets the company's hiring bar - while also having enough nuance that there's an opportunity for absolutely great candidates to impress me.

And the problem has to be possible to state unambiguously. Can't have a candidate solving the problem, but failing the interview because there was a secret requirement and they failed to read my mind.

And of course, if we're doing it in person on a whiteboard (do people do that these days?) it has to be solvable without any reference to documentation.

Re: AI killed the tech interview. Now what?

#218

Post by itomato »

_the_inflator wrote: post #75
micheles wrote: post #61

Nowadays I am on the other part of the fence, I am the interviewer. We are not a FAANG, so we just use a SANE interview process. Single interview, we ask the candidate about his CV and what his expectations are, what are his competences and we ask him to show us some code he has written. That's all. The process is fast and extremely effective. You can discriminate week candidates in minutes.

Do you reevaluate them in predetermined intervals to see how your initial expectation matches the outcome?

With each Sprint, presumably.

Re: AI killed the tech interview. Now what?

#219

Post by tempoponet »

noirbot wrote: post #148
solumos wrote: post #65

Earlier quoted context omitted.

I don't code much outside of work. I have hobby projects from 10+ years ago, but they're not much more than landing pages copied from templates and wordpress installs. I mostly work in backend/data/platform engineering professionally. If I were asked to make a small project over a weekend, I'd be likely to decline rather than doing a more standard interview, or I'd use AI to do it in a reasonable timeframe (which see…

This points to another issue - when I do code outside of work, it's often specifically to try out things I don't do at work. After a day of doing backend work, I'll maybe put together a basic web UI for something. That code is likely awful because I just need it to be functional more than good, and also probably not related to the work I'd be hired to do. My most recent real "side projects" are a terrible OSS monte c…

This relates to another issue of using people's public github as a hiring signal. I don't share any of these repos because at a glance the code is ugly, broken, incomplete.

Below the surface, I'm probably scratching a very interesting itch. Exploring a specific idea or problem, and then I stop when I get my answer.

Post reply on HN