Live data from Hacker News

The Machine Learning Software Engineering Interview

eng.lyft.com

71–80 of 81 posts

Re: The Machine Learning Software Engineering Interview

#71

Earlier quoted context omitted.

I think you underestimate the level of software architecture and engineering skill that people with formal training in graduate level statistics bring to these jobs. I manage a team of machine learning engineers in a mid-size ecommerce company and I can tell you that the same person who is optimizing Dockerfiles for better layer reuse & figuring out how our CI pipeline will safely get secrets needed to retrieve model…

Your definition of ML engineer comes off as being a high-risk individual to have in an organization. I would rather split those into two separate orthogonal roles and have redundancy in my resource pool. It seems like a very difficult and scare resource to hire. How can Average Corp even think of hiring someone like that? Dead no from me.

The problem is you can’t really split them into two. You need the same person designing models to also be making tactical decisions about software design and production lifecycle because those things are almost always highly dependent on nuanced specifics of the model itself.

There is not a division of labor where one person makes the model and then throws it over the fence to a team that manages its production usage and lifecycle. That team over the fence would not be able to do it, and I’ve seen this attempted org structure fail hard everywhere its been tried for ML services.

What’s telling is that you bring up the personnel risk but you don’t consider the value add. When Average Corp hires someone, it’s because that person brings more value than they cost, period. It’s not because the person is “not risky” in some vacuum of decision making like you’re painting it out to be.

> Dead no from me.

That’s fine and all, but it usually indicates tech death of a company, and likely you have brain drain in more area than just machine learning. If you aren’t willing to take the risk and do what’s needed to structure the work and job to extract value from high performers, that’s a sign of corporate mediocrity and I think anyone with the skill set to be an ML engineer like this would already not even be applying to work in a place like that.

Honestly the risks are even worse than your comment says. There are also big risks around keeping this person intellectually engaged, giving them valuable job experience.

You pretty much have to pay them a lot, give them good work life balance, give them budget for conference travel & continued learning, give them meaningful upward career & compensation growth, and give them meaningful projects.

I look at this and think, yes, if the business can’t give all those things and still be coming out ahead on the person’s productivity, then you don’t want an ML engineer.

But more often the company needs an ML engineer and absolutely would gain more from their productivity than they lose on supporting all those job quality aspects — yet managers just take superficial offense at these demands and balk at the idea that you have to provide meaningful projects and career growth instead of just barking orders and expecting them to put up with work that does not help them grow.

Re: The Machine Learning Software Engineering Interview

#73

Earlier quoted context omitted.

Calm down. Machine learning is a part of software engineering. Like multiprocessing, computer graphics or network protocols. It is here to stay. It is a part of a pallete of algorithms with which one can build software.

Your comment is absolutely correct but further points out just how far astray data science has become from any meaningful work. This issue is that a huge number of "data scientists" have limited programming ability and nearly zero engineering sense. As a perfect example of this is the trend in most places I've seen where data scientists strive to increase the complexity of their model (so they can prove how "smart" t…

What are these data scientists? Most statisticians I know would just use the linear regression unless they needed a neural network for marketing purposes. Statisticians will spend years studying linear regression and variations in graduate school. I thought it’s a CS guy who would be more fascinated with neural networks.

Re: The Machine Learning Software Engineering Interview

#74

Earlier quoted context omitted.

What point is Graeber trying to make in Debt ? It seems to be “capitalism bad” but that may be too kind to the book’s coherence. On Bullshit https://en.wikipedia.org/wiki/On_Bullshit https://noahpinionblog.blogspot.com/2014/11/book-review-debt... > Now, this may sound a little silly - if someone wrote a book called "Metal: The First 5,000 Years," and then filled that book with stories of war and bloodshed, never fail…

> It seems to be “capitalism bad” but that may be too kind to the book’s coherence. have you read the book? The book is an exploration, and an interrogation, with so much to learn from that to say that about it seems pretty philistinic. Maybe you were just summing up the review you linked from Noah Smith. I read most of it, it's a bit meh but Noah doesn't really seem to be trying too much in it. This though: "leftist…

Shit, I remember most of the "bad stuff" in Debt predating capitalism by somewhere between centuries and millennia. Seems like a weird way to write it if its Secret Purpose was to be a long-winded hit piece on capitalism.

Re: The Machine Learning Software Engineering Interview

#75

Earlier quoted context omitted.

Your comment is absolutely correct but further points out just how far astray data science has become from any meaningful work. This issue is that a huge number of "data scientists" have limited programming ability and nearly zero engineering sense. As a perfect example of this is the trend in most places I've seen where data scientists strive to increase the complexity of their model (so they can prove how "smart" t…

What are these data scientists? Most statisticians I know would just use the linear regression unless they needed a neural network for marketing purposes. Statisticians will spend years studying linear regression and variations in graduate school. I thought it’s a CS guy who would be more fascinated with neural networks.

Well there are some fairly distinct camps forming in data science. You are correct that those coming from a statistics background would generally prefer simpler, more parsimonious models. There is a not-insignificant group that seem to be coming into the field via other channels (CS, boot camps, self-teaching, etc.) who view statistics as a field as a bit of a dinosaur and therefore the statistician mindset to be backwards. Simpler models aren't a good thing, they are a bad thing. Any amount of increased complexity is worth even a small amount of improvement in performance.

I think some of this is exacerbated by modern pillars of machine learning and data science. Competition sites like Kaggle are entirely based on maximizing test set accuracy, and so winning submissions these days are huge morasses of ensemble methods that are trained for days and weeks on GPUs, but in the end they are often only marginally better than some of the fairly basic standard approaches. And when companies like Google are building their bots for Go or Starcraft, they are using cutting edge techniques. When people see that and get inspired to get into data science, thats what they want to do, even the the majority of problems are more rooted in data quality, thoughtful understanding of the problem, and more rudimentary methods.

Its also the result of some of the rhetoric of important figures in the field. Yann LeCun has pushed back strongly in the past on criticisms of modern day machine learning's occasionally lack of concern with introspection and model understanding. Judea Pearl, a Turing award winner for his work in machine learning, devotes large portions of his pop-sci The Book of Why attacking the field of statistics on the whole, as well as engaging in multiple attacks on historical influencers in the field with such ferocity it borders on character assassination. He has even rebuffed modern critics, such as the very widely respected Andrew Gelman, by saying they are "lacking courage" by failing to accept his "revolutionary" causal inference methods over the traditional ones used in statistics.

The attitude is driven a lot by the people and institutions at the top, and as someone in the field, I unfortunately encounter this kind of thinking way too often.

Re: The Machine Learning Software Engineering Interview

#76

> In the context of the modeling onsite, we ask open-ended problems with sufficient business and problem context such that the candidate can clearly identify an ML-based approach to solve it. I'm disappointed the author wasn't more specific about where the line is drawn between "ML SWE" and "Research Scientist"/"Data Scientist" when it comes to the core ML competencies like model selection, evaluation, and design. Ha…

Undergrad here. This is kind of disappointing. I really want to work in this field, but it seems I must have a PhD at minimum and not sure if I want to take that step yet.

You don't, though a MS may be borderline necessary. Without a PhD you will likely only lose out on some jobs at companies that are doing hardcore ML research or are simply more invested in the DS hype than actually looking to solve problems with DS. There are plenty of companies out there with data science needs that can be filled by non-PhD data scientists.

Re: The Machine Learning Software Engineering Interview

#77

Earlier quoted context omitted.

What are these data scientists? Most statisticians I know would just use the linear regression unless they needed a neural network for marketing purposes. Statisticians will spend years studying linear regression and variations in graduate school. I thought it’s a CS guy who would be more fascinated with neural networks.

Well there are some fairly distinct camps forming in data science. You are correct that those coming from a statistics background would generally prefer simpler, more parsimonious models. There is a not-insignificant group that seem to be coming into the field via other channels (CS, boot camps, self-teaching, etc.) who view statistics as a field as a bit of a dinosaur and therefore the statistician mindset to be bac…

Thanks for sharing your expertise. It was very interesting to hear your perspective.

Re: The Machine Learning Software Engineering Interview

#78
post #32

Earlier quoted context omitted.

Heh, given that I am starting to see more and more companies that offer ML engineers $2-6k/month (before tax), it's starting to resemble gaming industry in all its negative characteristics instead.

I cannot tell from your comment whether 2-6k/month before tax should be considered a lot or a little. I think in the major tech centers that 2-6k/month is quite low for anyone with significant experience (>5 yrs). Do you disagree?

[deleted]

Re: The Machine Learning Software Engineering Interview

#79
post #57

Earlier quoted context omitted.

What's OR?

What's OR? https://en.wikipedia.org/wiki/Operations_research Basically a mathematical approach to problems of logistics and scheduling developed first in WW2. Very powerful in the domains for which it was developed but less generally applicable than enthusiasts hoped, leading to the usual “hype cycle”. If you have a problem OR could solve or just want to fool around with it PuLP is very easy to use https://pythonhost…

There is also Google OR tools.

https://developers.google.com/optimization

Re: The Machine Learning Software Engineering Interview

#80
post #52

Earlier quoted context omitted.

Calm down. Machine learning is a part of software engineering. Like multiprocessing, computer graphics or network protocols. It is here to stay. It is a part of a pallete of algorithms with which one can build software.

Yes! It's one tool in the software engineering toolbox! It's a great tool for some problems! Due to the hype it becomes a goal in some organizations however. "We need to do machine learning because we have big data" or some such. Doesn't matter if the problem could've been solved in 5% of the time and cost with 20 lines of code, thou shalt use machine learning. It doesn't help that data scientists (creating and train…

To me, it seem to be larger than one tool. I think of it as a color in a pallete, with which one can paint software. Octarine.

To put this statement into context, I'm speaking as someone who had been writing code in C, from the era of PC XT. Perhaps NIPS 2010 was a rite of passage to ML for me. There is a screen, full of industry grade C++ and PyTorch in front of me, right now...

Post reply on HN