Green Lumber Fallacy in Software Engineering

I know I'm being trolled but I'll try a serious answer.

Lots of people get good grades in data structure and algorithm courses where the test basically looks like these interviews. You get some problem that "asks" for some algorithm to be used, you implement it, done. Out of those people, maybe 10%-20% are going to be great programmers. IMNSHO. And I'm probably being generous. How do I know? I hire those people. Everyone I hire has great grades in those courses. And some of them, many years from now, are gonna be awesome. And I don't hire everyone that has good grades.

Most of those are exactly the people you're complaining about and they'll ace your interview. Why are you asking them a dynamic programming question and then complaining about their usage of Zookeeper or whatnot? These things have nothing to do with each other.

I didn't say those are the questions asked in FAANG interviews.

I think if ML is your domain, and you present yourself as an expert in this stuff, then technical questions along those lines are fair game. And you're right that you should know your stuff. And you should also have the ability to work in a different domain. I've written Huffman encoders a few times in my life (as far as I can recall never in a work setting) and I could maybe write one in an interview. But I can't write an arithmetic coder in an interview without looking it up. Can you? I mean what exactly is the point here? Sure, the stuff you do day in and day out, you should demonstrate that you're able to do it. The stuff that you don't do, you should demonstrate that you're able to understand this is another domain, research, and then do stuff.

This is a little bit like comparing PhDs to people without PhDs. If you're a machine learning PhD you will know a ton about the domain (and also you're forced to learn a ton about some adjacent domains). Are you a better programmer or software engineer? I don't think so. It's like comparing a mechanical engineer to a Physics PhD.. these guys are not interchangeable. CS PhDs, those guys with the knowledge you seem to think is that important on the top of their mental stack, tend to build those terrible pieces of software you're complaining about. (obviously can't generalize, some are also awesome engineers, just like some Physics PhDs might be great at machine design). Don't get me wrong, I have the utmost respect to PhDs and I worked with some brilliant scientists in different domains. I don't want most of them to write software ;)

First off, you are going off on a tangent with questions that are never asked in a FAANG junior engineer interview. I should have just ignored that tangent, but whatever.

Getting back to the point, the FAANG interviews focus on basic graph algorithms etc. Getting hired at a FAANG doesn't imply you are a great engineer. Thats not something you can figure out in 5 interview loops. It means that you have the tools to understand basic CS201 concepts and are not the "I will look up the algorithm when I need it" engineer - which is absolutely a recipe for disaster.

And these algorithms are very relevant, Apart from top sort, bloom filters etc. I have also used dynamic programming on the job, to tokenize product titles to minimize the entropy of the final inferred tokenization of a product, before indexing. I can't even begin to imagine working with an engineer who cant understand that a correct naive tokenization is exponential, that the standard trick to solve such a problem is DP etc. These are basic, it doesn't mean you are a great engineer. It means that you have the basics covered and don't need to be hand held through the implementation details of systems. It's the difference between telling an engineer "we use DP to minimize the tokenization entropy" and sitting with him/her for 1 hour and walking him through every step, like it is some kind of magic. The standard FAANG interview doesn't even cover probability that well, so they have a fairly lenient expectation from the engineers about the concepts that they need to know.

As for why I am complaining about zookeeper. Because zookeeper is an implementation of PAXOS algorithm which has an extremely high penalty for writes. A person who thinks, everything is a black box and he doesn't need to know about algorithms, and will "look them up" when required, is actually never going to looks up any algorithm. Like I said he won't understand something is a graph problem even when it is staring him in his face. I have given several examples of graphs, DP etc being used on the job. And you completely ignore the basic problem with guy implementing lost updates with race conditions.

1. He doesn't realize this is a standard problem, covered in a FAANG system design interview.

2. He doesn't look up standard solutions and cooks up his own hacks, using a central database to record state. His hack has more race conditions.

3. I have to tell him the standard solution to this is locks

4. He ignores my suggestion, and finds out about zookeeper. He thinks it's a key value store like Cassandra. He doesn't realize that zookeeper is an implementation of PAXOS and has very high write cost. Because, he doesn't care about algorithms. Everything is a black box to be glued together.

5. He judges good software engineering on the basis of object hierarchy design, design patterns etc. Algorithms, data structures etc are irrelevant to the job, except for vectors and hash maps. There are API services to do everything else.

This was an engineer at a non FAANG company. Having worked at both FAANG and non FAANG, my observations

1. FAANG engineers are generally higher quality and way faster at execution.

2. Excellent non FAANG engineers are plenty, but at a non FAANG they are fewer to be found. Very often, they end up at a FAANG a few years down the line.

3. FAANG hires don't necessarily have large scale system design skills. The interview cant really pick that up. That's what the promotion and annual reviews are for.

4. The FAANG loop is a classifier, just like any other interview loop - with both type I and type 2 errors. So, it will both reject good candidates and accept bad ones. Any criticism of FAANG interview loops that don't understand the concept of type I and type 2 errors are just emotional hyperventilation. None of the alternative proposals even care to show how it could possibly be better than the FAANG loop.

I think you are referring to Professional Orders (https://en.wikipedia.org/wiki/Professional_order) and, depending on where you live, there may actually be one. In my country, computer engineers have a professional order they can join after graduation, in the same exact way all the other kinds of engineers they've studied with do. Unlike some other engineering fields, the order does not enforce nor require active membership in any order to work.

There've been a few attempts to get this going. So far none of them have worked (see also: any kind of union specifically for tech workers).

The other professions have a "closed shop" - you cannot work in that profession without joining that association.

e.g. in the Anglosphere, you cannot practice as a lawyer without being a member of the relevant Bar Association, and to join that you must "pass the bar" which is an exam.

If we had this in Dev, then we wouldn't be able to write code for money until/unless we were a member of the Software Dev Association, and they wouldn't accept us until/unless we'd passed a rigorous exam (passing the bar is something that takes years of study) to prove that we could code. Then we wouldn't be facing ridiculous "but can you actually code?" tests during interviews.

But creating that closed shop has always failed (so far). Employers don't want it, and new coders don't want it. It's only us old hands who expect to be grandfathered into it that kinda like the idea.