Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to conduct a good programming interview

lihaoyi.com

Post reply on HN
Print view View original

171–180 of 188 posts

Re: How to conduct a good programming interview

#171

Post by balls187 »

tptacek wrote: post #168
balls187 wrote: post #143

Earlier quoted context omitted.

Other than the pain of being rejected (or rather the lack of validation), where does this reliability problem manifest? Elite tech companies have no problem with the current implementation of technical interviews. Engineers whos skills range from Great to Mediocre don't have a problem finding a job. Software quality has generally improved or stayed the same.

This is demonstrably false. Lots of great engineers are terrible at interviewing. If you ask around, you can get even some very high-profile devs to tell you stories about why they're not working at Google or Facebook or Amazon because of how badly their interviews went. Equally importantly: lots of terrible engineers are great at interviewing, and most people in the industry have had the experience of working alongs…

> lots of terrible engineers are great at interviewing and most people in the industry have had the experience of working alongside someone who is failing up, 2-year-stints at a time.

You mean lots of companies are terrible at firing terrible engineers.

The argument is basically the current implementation leads to false negatives. What impacts does that have?

Presumably those very high-profile devs aren't lacking employment opportunities, even if it isn't at the elite tech companies.

Companies, from Elites down to pleb-tier, while perhaps pressed for resources, are still able to execute.

So why is the problem of false negatives a problem we should focus on?

Re: How to conduct a good programming interview

#172

Post by balls187 »

lewisl9029 wrote: post #158
balls187 wrote: post #143

Earlier quoted context omitted.

Other than the pain of being rejected (or rather the lack of validation), where does this reliability problem manifest? Elite tech companies have no problem with the current implementation of technical interviews. Engineers whos skills range from Great to Mediocre don't have a problem finding a job. Software quality has generally improved or stayed the same.

> Engineers whos skills range from Great to Mediocre don't have a problem finding a job. How are you defining "skills" in this case? "Skills" as in the skills needed to perform well in the traditional technical interview gauntlet? If that's the case then you have a tautology on your hands. And if not, then maybe it'd serve us better as an industry to try to find a better proxy than the traditional technical interview…

Skills: writes code that's clean, efficient, simple, etc

My question is this: Why does the industry need to change the interview process?

If companies are able to maintain a quality bar of it's employees, and engineers are able to find meaningful employment, doesn't that mean the system, even if it has a flaw (false negatives) isn't really a problem?

Perhaps the problem is that many of us who write code, consider ourselves good at it, and getting rejected is a hit to our ego. We want to change how interviews are done solely for the benefit of our egos, when instead, we should understand that if you interview, you might get rejected, and that is okay.

Nothing is fundamentally wrong with the process, nothing is fundamentally wrong with you, and in 6-12 months, you can try again.

Re: How to conduct a good programming interview

#173

Post by SilasX »

kevmo314 wrote: post #45
baddox wrote: post #38

Earlier quoted context omitted.

I discussed it briefly. My interviewer was unpersuaded, and the other interviewer in the room (who wasn't performing this interview) suggested we just move on. I believe the unconvinced interviewer was simply a very junior developer and an even more junior interviewer. He seemed quite nervous.

One of the worst interviews I've had the interviewer was not only similarly incorrect, but then afterwards proceeded to try to give me advice on interviewing skills, and how I should've "taken his hint" and been "more amenable to other answers". I probably would've been open to alternate answers if they were, y'know, factually correct.

If you think that's bad, I was this guy[1].

tl;dr:

1) Interviewer insists that an operation must be n^2 "because it has a nested loop" even though the nested loops are only iterating over their own subset of the master string.

2) Interviewer refuses literally any test that would help decide which of us is right.

3) Interviewer turns out to not even understand the concept of resolving disagreements by appealing to agreed-upon facts, insisting that he would only be convinced by the absence of a nested loop (i.e. talking about some other algorithm).

4) Interviewer is five-year backend lead.

5) Interviewer vetoes me from the rest of the process purely because of that disagreement.

6) Because I thought I was going insane, I implemented it at home and ran the problem with different sizes and the run time with up as O(n) like clockwork.

[1] It's okay, I'm fine with people knowning and have outed myself elsewhere: https://news.ycombinator.com/item?id=6070001

Re: How to conduct a good programming interview

#174

Post by tptacek »

balls187 wrote: post #171
tptacek wrote: post #168

Earlier quoted context omitted.

This is demonstrably false. Lots of great engineers are terrible at interviewing. If you ask around, you can get even some very high-profile devs to tell you stories about why they're not working at Google or Facebook or Amazon because of how badly their interviews went. Equally importantly: lots of terrible engineers are great at interviewing, and most people in the industry have had the experience of working alongs…

> lots of terrible engineers are great at interviewing and most people in the industry have had the experience of working alongside someone who is failing up, 2-year-stints at a time. You mean lots of companies are terrible at firing terrible engineers. The argument is basically the current implementation leads to false negatives. What impacts does that have? Presumably those very high-profile devs aren't lacking emp…

Almost every company is terrible at firing ineffective engineers. Even some of the companies that are famous for doing so ("'meets expectations' earns a generous severance package!") don't necessarily manage to accomplish that in less than a year. This stuff sounds very simple on message boards, but is much harder to do in real life.

The first N months of a developer's tenure at a job is difficult to measure because of ramp-up and calibration (even good developers might hop to different teams after getting hired based on work style fit issues). Whole projects can flounder and need rescuing so that attention to ineffectiveness gets deflected to group leads. Most good development shops will try to make things work for a new hire that isn't performing at the 6 month mark, and that process takes a few months.

Once the decision is made to fire, most big companies will PIP rather than fire. Sure, once you've hit the 7-8 month mark of poor performance, your severance from the firm may be a fait accompli. But the ultimate process to make that happen can easily be made to take more than 12 months.

Once you have a year-long stint at a company on your resume, you can bank that "experience" on your resume.

And that assumes a relatively well-run companies. Most companies aren't well run in this regard, and you can be a terrible developer but last 2+ years in most jobs. If you can reliably put more 2+ year jobs on your resume than 1-year jobs, you will end up with what will appear to most companies to be an extremely attractive resume.

Remember, many management-track developers only write code for ~8-10 years. Once you fail up to Dir/Eng, you're basically tenured.

Since almost nobody qualifies candidates effectively, failing up is a super powerful strategy for this field.

Re: How to conduct a good programming interview

#175

Post by tptacek »

thaumasiotes wrote: post #146
tribythrowaway wrote: post #100

Earlier quoted context omitted.

Triplebyte's flaw is like that of many other take-home tests: they favor people who lied about how long it took. When I did it, I had to implement a regex parser. I knew intuitively that the state machine approach was best, but if I read up on state machines and coding them, I would bust the 3 hour limit, so I did the best I could to deliver something working within the three hours. Feedback: "Obviously the state mac…

The state machine approach can't handle perl-compatible backreferences (since true regular expressions can't use them), which apparently some people want. That's feedback you can defend yourself from to some degree. When I implemented a regex parser for Triplebyte, the time limit was much longer than three hours, and I did use state machines. The feedback I got was "we thought you wrote a great, full-featured regex p…

You can already see that this is a broken technical qualification process. If you can "defend yourself" from a demerit on the challenge's rubric, you have a subjective process. The smooth talkers will glide past regardless of their abilities, and the gifted developers that don't have interview skills will get washed out by it at random.

If there's a point to outsourcing technical qualification, it's exactly to avoid this kind of problem!

Re: How to conduct a good programming interview

#176

Post by balls187 »

tptacek wrote: post #174
balls187 wrote: post #171

Earlier quoted context omitted.

> lots of terrible engineers are great at interviewing and most people in the industry have had the experience of working alongside someone who is failing up, 2-year-stints at a time. You mean lots of companies are terrible at firing terrible engineers. The argument is basically the current implementation leads to false negatives. What impacts does that have? Presumably those very high-profile devs aren't lacking emp…

Almost every company is terrible at firing ineffective engineers. Even some of the companies that are famous for doing so ("'meets expectations' earns a generous severance package!") don't necessarily manage to accomplish that in less than a year. This stuff sounds very simple on message boards, but is much harder to do in real life. The first N months of a developer's tenure at a job is difficult to measure because…

Why are the terrible engineers terrible, leading to them being fired (after 1-2 years?)

I've not hired anyone who turned out to be bad at coding, but I have hired people who had other performance related issues, which were difficult to suss out during an interview.

From my own personal performance, my approach to interviewing has led to far more true positives, than false positives, so I am biased towards continuing.

Re: How to conduct a good programming interview

#177

Post by crzwdjk »

balls187 wrote: post #143
tptacek wrote: post #67

It's good to think carefully about how you interview candidates. Most of the ideas in this post are good. You should definitely let candidates type answers on computers. You should definitely work to put them at ease, in an environment that approximates their actual working environment. But really, this is window dressing. The problem is that technical interviews just don't work reliably. We have better ways to quali…

Other than the pain of being rejected (or rather the lack of validation), where does this reliability problem manifest? Elite tech companies have no problem with the current implementation of technical interviews. Engineers whos skills range from Great to Mediocre don't have a problem finding a job. Software quality has generally improved or stayed the same.

Anecdote, but a telling one: in my previous round of job-hunting, I had a success rate of exactly 0% with whiteboard coding interviews and a success rate of 100% with other interview methods. That's a pretty stark correlation.

Re: How to conduct a good programming interview

#178

Post by Corrado »

tptacek wrote: post #107
michaelchisari wrote: post #104

Earlier quoted context omitted.

This is my approach for evaluating new candidates as a first step. For front-end, for instance, I start with a codepen that has the basic skeleton that includes the libraries we use internally, has a few html elements styled and placed, and the required javascript boilerplate. Then I provide 10 tasks that are written much in the way the company would put tasks in asana or git or jira. Each task builds on the previous…

A huge advantage of this approach is how much more confident you can be that you've qualified candidates technically. You can --- and should --- exploit that confidence commercially. Change the posture of your outreach to reassure people that you're interested in applicants regardless of their background or resume. Change where you run job ads and how you word them. If you have a strong qualification process, your jo…

This is some pretty outstanding insight. I work with middle school kids to get them excited about computers and programming and it's amazing how many of the ones that "get it" are not the ones you would pick. Frankly, I think they are surprised at how much they like it and you can almost see the light bulb go on. If we could reach those type of people and get them to interview it would almost certainly change the industry for the better.

Re: How to conduct a good programming interview

#179

Post by Jhsto »

balls187 wrote: post #176
tptacek wrote: post #174

Earlier quoted context omitted.

Almost every company is terrible at firing ineffective engineers. Even some of the companies that are famous for doing so ("'meets expectations' earns a generous severance package!") don't necessarily manage to accomplish that in less than a year. This stuff sounds very simple on message boards, but is much harder to do in real life. The first N months of a developer's tenure at a job is difficult to measure because…

Why are the terrible engineers terrible, leading to them being fired (after 1-2 years?) I've not hired anyone who turned out to be bad at coding, but I have hired people who had other performance related issues, which were difficult to suss out during an interview. From my own personal performance, my approach to interviewing has led to far more true positives, than false positives, so I am biased towards continuing.

> other performance related issues

What are some examples of these?

Re: How to conduct a good programming interview

#180

Post by mlevental »

tptacek wrote: post #69
sillysaurus3 wrote: post #64

Earlier quoted context omitted.

Yeah, I'm a big fan of your approach. Unfortunately it seems quite difficult to get anyone to do it. Most companies have interview pipelines that are well-established and immutable, so only new companies are inclined to try it. And even Triplebyte unfortunately fell back on a combination of abstract coding tests and amorphous "cultural fit" criteria. It was a neat hack as a candidate to propose to the interviewing co…

I think asking candidates to do "real" work is bad for a bunch of reasons: * It's intrinsically exploitative, even when compensated, because real work done under similar terms is paid far more highly than prorated salary (it also does weird things to salary negotiation, by extracting an up-front concession on rate). * It requires different candidates to work on different problems, which means you're not getting a rel…

did you read the grandparent? you use the same bug by checking out a commit before the bug fix was applied.
Post reply on HN