Live data from Hacker News

The Machine Learning Software Engineering Interview

eng.lyft.com

11–20 of 81 posts

Re: The Machine Learning Software Engineering Interview

#11
post #8

This blog post was so painful for me to read. This is a symptom of "bullshit" going on around in big tech companies. "bullshit" here is an economic term defined in the book "bullshit jobs". https://www.amazon.com/Bullshit-Jobs-Theory-David-Graeber/dp... Reading through the post, I was noticing So much corporate Jargon which really does not mean anything important. Dehumanizing language when describing people intervie…

"Obfuscation" and "delusions of grandeur" are practically synonyms for ML and Data "Science" in this industry. I've been around for a while and I've never quite seen something as over-hyped and hyper-glamorized as these two specializations.

Re: The Machine Learning Software Engineering Interview

#12
post #2

It would be great if companies would allow you to bypass the code challenge if you grant them read git access to a relevant project that you have ownership of.

Problem is its easy to cheat about whats yours

I think the trick here is have people talk through what it does, how and why (as in what are the tradeoffs, what other approaches could have been taken, etc).

You can't fake this. Or to put it better, even if the code isn't yours and you can do this well, it doesn't even matter that the code isn't yours as in doing this you by definition have the skills and knowledge to reimplement it anyway.

Re: The Machine Learning Software Engineering Interview

#13

> In the context of the modeling onsite, we ask open-ended problems with sufficient business and problem context such that the candidate can clearly identify an ML-based approach to solve it. I'm disappointed the author wasn't more specific about where the line is drawn between "ML SWE" and "Research Scientist"/"Data Scientist" when it comes to the core ML competencies like model selection, evaluation, and design. Ha…

I think you underestimate the level of software architecture and engineering skill that people with formal training in graduate level statistics bring to these jobs.

I manage a team of machine learning engineers in a mid-size ecommerce company and I can tell you that the same person who is optimizing Dockerfiles for better layer reuse & figuring out how our CI pipeline will safely get secrets needed to retrieve model files for integration tests is the same person researching new variations of triplet loss in a paper from arxiv they present in our team’s journal club & developing Bayesian hierarchical regression models and explaining why partial pooling using an industry-specialized prior distribution actually produces coefficients in the model fit that have a meaningful improvement over OLS for some business outcome.

The same people that are refactoring a collection of unit tests to be parallelizable and save us 45 seconds on every test run are also running huge hyperparameter tuning experiments to get a learning rate scheduler that allows us to reduce training time for an in-house deep GRU neural network from 24 hours to 15 hours, and they are defining infrastructure as code tooling to even create the very GPU environments where this training is taking place.

The skill set really is extremely different from general backend engineer with a proclivity for hacking around ML models and also is really different from “research developer” who rarely deals with end to end systems or necessary concerns of production or quality code factoring, and also is super different from data science which is effectively just ad hoc business analytics but with more impressive pedigree on the resume.

I’d define a machine learning engineer as taking someone with several years of graduate experience in statistics / machine learning inclusive of formal probability theory, analysis, topology, Bayesian stats (or work + research experience that is equivalent, though it’s super rare for this to be able to replace formal graduate math training), and then adding senior level skill set in high-performance computing, generalist backend engineering, system architecture, and full management of the lifecycle of complex production systems.

The only piece of common modern engineering that I would say ML engineers typically don’t have a senior-ish level of command over is frontend development (though some do out of just hobbyist interest).

Re: The Machine Learning Software Engineering Interview

#14
post #8

This blog post was so painful for me to read. This is a symptom of "bullshit" going on around in big tech companies. "bullshit" here is an economic term defined in the book "bullshit jobs". https://www.amazon.com/Bullshit-Jobs-Theory-David-Graeber/dp... Reading through the post, I was noticing So much corporate Jargon which really does not mean anything important. Dehumanizing language when describing people intervie…

I think you are confusing your criticism of the author's language and the "usefulness" of the actual job.

I don't see how a software engineer with a focus on machine learning is a "bullshit job".

Re: The Machine Learning Software Engineering Interview

#15
post #3

Enjoyed the post but I found this sentence interesting. >As a candidate, it’s easy to get a foot in the door and be evaluated by an interviewer. Do people really feel that way about a ML SWE postion at Lyft? I would be interested to see what percent of applications they receive make it to the door.

The article does hint at the percent: “YMMV: healthy numbers for an interview loop for the phone screen to onsite, to offer, and to offer acceptance are, for example, 50%, 25%, and 70%.”

Re: The Machine Learning Software Engineering Interview

#16
post #2

It would be great if companies would allow you to bypass the code challenge if you grant them read git access to a relevant project that you have ownership of.

This is a common sentiment among applicants, but after many years of hiring people, just FYI, looking at people’s github projects is one of the lowest quality signals I get from an applicant. It takes me forever to wade through someone’s code to figure out what they did or how good they are, and when there are multiple people on the project, it’s not easy to see who’s driving or what the cooperation/conflict situation is by looking at the git history. It can also backfire in a variety of ways you might not want and not know about, so keep this in mind. Your personal project code sometimes has lots of decisions in it that don’t look particularly good to a very experienced engineer. I can’t even count how many people have sent me github links for code they’re proud of that when I study it makes me second-guess the candidate a little bit. As an interviewer, I honestly prefer to hear the candidate talk through their code so I can find out they’re learning and hopeful rather than read code that says more about today’s limitations than tomorrow’s potential.

Re: The Machine Learning Software Engineering Interview

#18

> In the context of the modeling onsite, we ask open-ended problems with sufficient business and problem context such that the candidate can clearly identify an ML-based approach to solve it. I'm disappointed the author wasn't more specific about where the line is drawn between "ML SWE" and "Research Scientist"/"Data Scientist" when it comes to the core ML competencies like model selection, evaluation, and design. Ha…

I think you underestimate the level of software architecture and engineering skill that people with formal training in graduate level statistics bring to these jobs. I manage a team of machine learning engineers in a mid-size ecommerce company and I can tell you that the same person who is optimizing Dockerfiles for better layer reuse & figuring out how our CI pipeline will safely get secrets needed to retrieve model…

I don't think I'm underestimating "people with formal training in graduate level statistics", but quite possibly the particular people you think of when you say the quoted phrase.

I've worked with stats/comsci/bio-informatic/ML PhDs at multiple multi-billion-dollar software companies now and it's certainly not true that even the majority of people with graduate stats training are excellent software engineers. Would love to work at a company where that's true, I just don't think that's the common case at all.

> The skill set really is extremely different from general backend engineer with a proclivity for hacking around ML models and also is really different from “research developer” who rarely deals with end to end systems or necessary concerns of production or quality code factoring, and also is super different from data science which is effectively just ad hoc business analytics but with more impressive pedigree on the resume.

Totally agree. The terms are very messy though. The "ML Engineers" with PhDs in ML are still called "Data Scientists" at Zendesk, for example. At other companies "Data Scientist" just means 'Data Analyst that knows SQL'.

> I’d define a machine learning engineer...

Would be happy with this definition. It's much clearer than what you can gather from Lyft's post. It's close to the definition I currently hold to. On this definition I've enrolled in graduate Maths+Stats as I'm still almost entirely a Data Engineer / Backend Engineer in terms of skillset.

Re: The Machine Learning Software Engineering Interview

#19
post #16
post #2

It would be great if companies would allow you to bypass the code challenge if you grant them read git access to a relevant project that you have ownership of.

This is a common sentiment among applicants, but after many years of hiring people, just FYI, looking at people’s github projects is one of the lowest quality signals I get from an applicant. It takes me forever to wade through someone’s code to figure out what they did or how good they are, and when there are multiple people on the project, it’s not easy to see who’s driving or what the cooperation/conflict situatio…

I had to read through your comment twice to get it. My experience leads me to agree with your idea of letting people talk through their own code. That category of conversations has usually been very rich and telling, much better than one over imaginary binary tree rotation algorithms or a "design a url shortener"-ish question. My favorite part is where a skillful candidate can lead the interviewer to interesting and real problems and solutions, demonstrate ability and both get to enjoy the conversation.

Re: The Machine Learning Software Engineering Interview

#20
post #8

This blog post was so painful for me to read. This is a symptom of "bullshit" going on around in big tech companies. "bullshit" here is an economic term defined in the book "bullshit jobs". https://www.amazon.com/Bullshit-Jobs-Theory-David-Graeber/dp... Reading through the post, I was noticing So much corporate Jargon which really does not mean anything important. Dehumanizing language when describing people intervie…

It seemed like a lot of words to say very little.
Post reply on HN