Silicon Valley's dirty secret - age bias

[Disclaimer: I'm an EIR at Sequoia at the ripe old age of 46]

Are you sure the proportion was the same? Reason I am asking is because for people who are bad at programming, there are various process/ management roles available that might better suit.

It seems rather impossible to me that someone would voluntarily continue programming for a couple of decades and not be excellent.

Beyond the 30's, I imagine people would be passionate about it to continue taking up programming roles. And this passion should translate into excellence.

Happy to be corrected if you have any data-points.

It's a hard balance. In the long haul, you often want people who are careful and diligent and have experience that tempers their urge to jump into things too quickly. But, this type of attitude can actually hurt in startups. In an interview where the particular 3 or 4 things you touch upon are extrapolated (correctly or not) to be the main self-identifying traits of how you work and how you think about problems, if you tell even just one story about superior databases from the 70's or seem to be overly-pedantic when it comes to blowing holes in ideas this can be a 'red flag.'

In a startup environment, I'd rather have the engineer who I think will fall in a hole 3 times and figure out a solution eventually than another who I think will think too hard and fall in one in 3x the time due to their fear built up from previous failures. In an interview, if I am spending my time with the person mostly talking about things that have gone wrong and how everything new is old, I will worry they will be too paralyzed to just hack on and break things -- and this type of focus, while obviously invaluable in many situations, tends to come from more experienced engineers since they have seen more shit go wrong.

The problem is when these problems re-appear, problems that you personally grappled with for many years, it's easy to just want to throw up your hands in frustration because you thought the issue was settled. But of course, in reality its not about the issue being settled or not, it's about the capability of technology evolving and macroscopic design patterns falling in and out of favor due to the particular tradeoffs available with the state-of-the-art. But it's hard to see this objectively when you had a lot of emotional investment into any problem, developed some expertise, and see your entire way of solving problems being thrown out for what you perceive as a fad.

A contemporary example that I've mentioned already is the recent abandonment of the relational model/ACID in databases. A cynic will tell you the people doing this are stupid because it's an inferior solution. But they fail to mention the reason people are doing this: it's because the relational technology has not caught up with the scalability requirements of many problems faced by engineers today. An experienced engineer with the right attitude can have an excellent perspective and hedge their technology bets accordingly: the people deploying NoSQL are not stupid, they are just getting their jobs done. But at the same time, this too shall pass, since the relational model and ACID are fundamentally good ideas, and eventually someone will swing the pendulum back that way once other problems are solved. So, for the experienced engineer who takes a longer view, certain types of NoSQL databases can serve a role but are likely going to be considered a transitional technology and systems should be architected accordingly. The inexperienced engineer will not recognize this and will go all-in with NoSQL and fall right into the traps that relational modelling and ACID are designed to solve. Lo and behold: Google Spanner paper is published -- but we're not on the other side of this cycle yet until this tech is commoditized.

This tendency of cynicism is particularly worsened because engineers are naturally repulsed by the idea of reinventing the wheel, particularly a wheel as giant as these "big problems I thought we solved already." It's also worsened by the fact that within any new technology cycle, there is often more noise than signal, and so it's easy to latch on to the noise as evidence that there is no signal if you're biased to believing it's all noise. It's also much easier to tell yourself that there is no new innovation happening, it's all noise, when the people building the new stuff are all younger than you and many seem to be unaware of history.