Live data from Hacker News

Why Triplebyte Failed

otherbranch.com

191–200 of 282 posts

Re: Why Triplebyte Failed

#191
> The idea was this: if higher-education and prestigious experience weren't reliable sources of quality, and if you could make an ML model that could identify it better through testing

The problem is that in reality those two sources of data are really reliable. YC itself is mostly composed of people (over 90%) from prestigious schools and organizations.

It’s rare to see anyone accepted to YC, who was from a mediocre or unknown institution for both education and work experience.

Re: Why Triplebyte Failed

#192
I always find it strange that people who talk about tech interviewing inexplicably overlook what seems to me to be a core defining characteristic: they are highly traumatic. You take some poor bastard and have him struggle at coding puzzles in front of someone he very much wants to impress and then watch as he fails miserably. They are left feeling like they are biologically inadequate to their job. It's a direct assault on egalitarian sentiment - a load bearing pillar of civilization - even if it is more or less a noble lie.

By definition, they only take the top 1%, and 99% of people get to eat shit. Inspiring existential resentment in the vast majority of people who interact with you is obviously not a recipe for good karma.

Re: Why Triplebyte Failed

#193

Earlier quoted context omitted.

Every airplane flies pretty much the same way, and pilots all get paid pretty much the same* Every website stack, and the level of complexity under it, is unique, and there's also a huge pay differential. *could be false assumption on my part

Maybe it’s time to start thinking about doing to software what has been done with other professional fields: licensing and checking out of various levels. If I have to spend 30 hours or so learning, practicing and demonstrating my knowledge about aircraft instrument procedures before I can attempt that as a pilot in real airspace, maybe it’s not that big of a jump that we’d license different software features, and go…

I think the hard thing is that there’s just a lot of mediocre programmers out there writing mediocre software. Should they be accredited or not?

I think a lot of average programmers will end up accredited if they see it as a path to a job, just like we see with Microsoft certificate programs. And if that happens, I wouldn’t want the accreditation test to be my company’s hiring bar. I’ll still run my own interview to make sure candidates are actually good. And so will most companies. So the time consuming interviews won’t actually go away.

The one big problem licensing would solve is that we could insist some baseline amount of security knowledge is in the tests. Right now, plenty of people get jobs handling sensitive personal data without knowing the first thing about how to keep data secure. That just can’t continue. It’s insane.

Re: Why Triplebyte Failed

#194

Earlier quoted context omitted.

To be clear: it was a bad decision both ethically and tactically, and I am not in any way defending it. I am explaining it, in the same way that one might explain why a bridge collapsed. (I will also note that one piece of the backlash somewhat misunderstood things - the profiles were "public" to subscribers, not to the internet at large, insofar as that distinction is meaningful.) The reason it was a consequence of…

A meaningful explanation of why a bridge collapsed would be: "Facing financial pressure, the board voted to re-open the bridge despite advice from its own experts that it was unsafe and faced a high chance of catastrophic failure." What I'm hearing instead -- is a categorical focus on "incentive pressures", rather than on the obvious lapse of sound judgement on the part of those responsible. As revealed, quite plainl…

If I gave that impression, I apologize. It wasn't my intent. So let me say clearly: it was a lapse of sound judgement, I disagreed with it both then and now, I've said so both publicly and internally since the day it happened, and I think a lot about how to avoid such errors myself.

I don't want to minimize the error. I want to show it in the light in which it was made, because no one ever comes up to you in business and says "hello, do you want to sell your soul today?". That's not what moral compromise looks like. It looks like trying to do a good job and not fully processing the decision you're actually making. That's an error that you make with normal human levels of normal human failing even though that error is abnormally costly.

To return to the bridge analogy for a second, the board that votes to re-open the bridge isn't sitting there going "hmm, should I kill fifty people today?". They've got a dozen people telling them a dozen things are immediate crises, and this one is particularly costly to listen to. So it's a little easier to believe that the civil engineer's report is just someone being alarmist than it is to believe that you've got a really hard decision to make, especially because you're not thinking about it too hard. And that's doubly true when you know it's likely to get you voted out of office, replaced by someone who cares about infrastructure less than you do, who definitely wouldn't close the bridge.

That doesn't make fifty people any less dead when the bridge collapses. But it does change the solution that helps you not collapse bridges, a solution that - as a newly-elected board member in this analogy - is a question that I am deeply concerned with.

I agree, fundamentally, with what you're saying: that you have a responsibility to overcome your incentives, that if you're one of those board members you have to get over your fears and close the bridge. I get that, and I agree with it. But to accomplish that requires understanding the kind of mindset in which one opens the bridge, which is usually not the mindset of deliberate malice. We don't have to compromise our moral principles or our understanding of right and wrong to understand the human weaknesses that lead people astray.

-----

I'll give you a concrete example from my recent history. On my very first sales call for Otherbranch, which did not go well, I asked the person I was talking to (who happened to have been a salesperson) for advice afterward.

His advice? Lie more.

Of course, he didn't say it in those words. He said something like "listen to your prospect's concerns and needs, and then talk about how you're a really focused solution to them". So if your prospect says they're concerned with candidate quality, you talk about how candidate quality is the focus of your process, the thing you're really specialized at. Or if your user says they're concerned with time investment, you talk about how that's the thing you're really specialized at. And so on.

In the one sense, this is totally expected behavior. Everyone already assumes a person on a sales call is already doing this. I would imagine that, to most people, this isn't really even a moral blip of significant size. And yet the advice fundamentally is "you should lie more", even if it's lying in this localized, normalized way.

Now, I don't do that. But I don't know how much that costs me. It almost certainly does cost me something, at least in the short term. That's a cost I am, provisionally, choosing to pay. But suppose that I knew for certain the decision was "if you don't do this, your business won't exist, and someone who does lie - a lot more than you do - will take your place". Would I be justified in bending to the fact that this is just the reality of doing business? I'm not sure I would make that argument, but it's not like a reasonable person couldn't make it. Is it better to be uncompromisingly moral and fail to be effective, or to win by being as bad as everyone else? It's not a trivial question.

One of the first things I wrote, when I started planning Otherbranch, was:

> Do not spin, do not mischaracterize, do not omit, do not grey-pattern.

That gets tested every single day. Every single day. It's tested on every call, every conversation, every sales pitch, every email, in ways large and small. I've been amazed at just how often I catch myself wanting to spin just a little teeny bit, because it's obviously the best approach to take in terms of getting Otherbranch off the ground. And if I think I'm a more ethical businessperson than most (and I do), isn't that better than the alternative? Those are the thoughts in my head.

And the thing is, I might be wrong about this. This might be fatal to running a company. It might just not be possible to win by those rules. I might have doomed my entire company and every bit of work I and others are putting into it when I wrote those words on like hour 60. And if I did, and the next person who comes along sees my failure and recognizes that fact from my next company postmortem, would you be able to blame them if they lied just a little?

I struggle with this kind of question a lot. Because I share your moral convictions and your belief that, ultimately, we need to call a spade a spade and call out when people are harmed or their autonomy disregarded. But I also want my moral convictions to have teeth, and that means not completely sabotaging my ability to get anything done. I don't think this is a particularly new moral conflict. I cannot possibly be the first person to try to navigate these waters, which should scare me even more, because it looks like the sharks ate the last guy. But I don't get a choice about navigating them if I want to get anything done.

Does any of this make any sense? This is already way longer than I'd intended to write and is definitely not my best work, but I'm trying to articulate something complex and personal because it's the only response I have to what you're saying. I think we agree on the moral principles, but I think I'm arguing for a nuance in their application that you aren't.

Re: Why Triplebyte Failed

#195
post #35

It's interesting to see how much "standardized cross-company interview for software eng" has been consistently a cursed problem in the industry. Unlike airline pilots (or I'm certain many other professions), every company in the valley insists on re-interviewing a candidate in their own custom and unique way, instead of trusting some sort of an industry-wide certificate only achievable through a standardized test. Wo…

If you create a standardized test it will be gamed. Even with the small modicum of standardization around interview questions that we currently see, people have published books like Cracking The Code Interview, making it easier for people who don’t have the skills for a particular job to pass interviews at any place that uses standard-ish questions. Furthermore, as an avowed enemy of “Clean Code”, I don’t want to see…

Why do standardized tests work for so many other industries?

Re: Why Triplebyte Failed

#196

Earlier quoted context omitted.

Every airplane flies pretty much the same way, and pilots all get paid pretty much the same* Every website stack, and the level of complexity under it, is unique, and there's also a huge pay differential. *could be false assumption on my part

> Every airplane flies pretty much the same way Not exactly, but that's why we have different certificates, endorsements, and type ratings that show demonstrated competence with each type of airplane.

A popular aircraft type is likely to be built for 20+ years and be flying for 40 or more. The 737 was rolled out in 1967, and the fourth generation is still being built. This is rather like a major chunk of the world's computing infrastructure running on Fortran (F77, F90... 2008, 2023).

Oh, wait, it does.

Re: Why Triplebyte Failed

#197

Earlier quoted context omitted.

I've never had two in-person interviews that were at all similar in any way. Maybe instead of a 'practice' interview people can just be told what to expect so they know how to prepare. Offering practice interviews just makes it sound like the interview is a performance.

The triplebyte interviews were pretty tightly scripted to try and remove interviewer bias. If you did two interviews with TB, you would have found them pretty similar. We also did tell candidates what to expect and how to prepare. Most candidates didn’t read our preparation notes. The people who did did better in the interview. (Source: I was one of TB’s interviewers.)

> Most candidates didn’t read our preparation notes.

looks at recent interviews

Ah, the more things change...

Re: Why Triplebyte Failed

#198
post #157

Earlier quoted context omitted.

Seriously, if you've never hired before, you have no idea how bad this can get. Here's [1] our practice coding problem. It's quite similar to the one we use on our interview, and not too far from the one Triplebyte used in the past (ours is tuned to be slightly harder at the beginning and slightly easier at the end). The vast majority of candidates, even with some reasonable pre-filtering, do not get past the first s…

That was a fun coding task. Minor bug in the illustration of Step 4: the mine count in second row should be 8 not 4

Thanks for the correction! Fixed.

Re: Why Triplebyte Failed

#199
post #196

Earlier quoted context omitted.

> Every airplane flies pretty much the same way Not exactly, but that's why we have different certificates, endorsements, and type ratings that show demonstrated competence with each type of airplane.

A popular aircraft type is likely to be built for 20+ years and be flying for 40 or more. The 737 was rolled out in 1967, and the fourth generation is still being built. This is rather like a major chunk of the world's computing infrastructure running on Fortran (F77, F90... 2008, 2023). Oh, wait, it does.

I'm not exactly sure what your point is, but it's even stronger when you include F'66 on that list.

Re: Why Triplebyte Failed

#200

Earlier quoted context omitted.

That's an interesting point, but then one wonders that if software eng are ultimately artists, why are we not having them work on their portfolios like the other art disciplines? Is that the fundamental problem?

> one wonders that if software eng are ultimately artists I find treating software engineering like art is a very dangerous approach and bound for failure. A lot of software engineering comes down to being comfortable with code and how the computer works. That has little to nothing to do with art.

A lot of painting still life comes down being comfortable with brushes and knowing how paint works. A lot of music comes down to know scales and chords work. Most art still requires fundamental mechanics.
Post reply on HN