Why we do machine learning engineering with YAML, not notebooks
towardsdatascience.com
Why we do machine learning engineering with YAML, not notebooks
1–10 of 31 posts
Re: Why we do machine learning engineering with YAML, not notebooks
#2> When I say production machine learning, I’m referring to machine learning that manifests as a product feature. For example, Uber’s ETA prediction, or Gmail’s Smart Compose.
You can bet that prod services from companies you heard of are running on something more analogous to versioned docker images. Not a yaml file which says, 'Go run whatever predict.py is in the current folder.'
The moment one of your dependencies breaks your code, or snookers your performance, there will be a lot of head scratching going on.
Re: Why we do machine learning engineering with YAML, not notebooks
#3The cortex tool mentioned looks really useful to get a service running out of a trained model. Though I didn't really understand what the article is trying to get at. Storing your deployment configuration in yaml and json files is pretty much the standard.
Re: Why we do machine learning engineering with YAML, not notebooks
#4Reads like someone was forced by their marketing team to write an article about anything at all.
Re: Why we do machine learning engineering with YAML, not notebooks
#5The title is seriously misleading. They aren't doing their ML engineering in yaml. If you look at the snipper the article, you can see that their code is in flat .py files. The config is in yaml (which is also how everyone else uses it). It's like someone saying that they do their ML in a dockerfile.
Re: Why we do machine learning engineering with YAML, not notebooks
#6The title is seriously misleading. They aren't doing their ML engineering in yaml. If you look at the snipper the article, you can see that their code is in flat .py files. The config is in yaml (which is also how everyone else uses it). It's like someone saying that they do their ML in a dockerfile.
The linked site (towardsdatascience) is kind of like medium blogs. High variability in quality with a lot of self-promoters and the occasional diamond in the rough. But a ton of rough.
Re: Why we do machine learning engineering with YAML, not notebooks
#7The cortex tool mentioned looks really useful to get a service running out of a trained model. Though I didn't really understand what the article is trying to get at. Storing your deployment configuration in yaml and json files is pretty much the standard.
Perhaps for young people it's actually necessary to mention JSON and YAML as you people tend to read the news rather than history or best practises textbooks?
Re: Why we do machine learning engineering with YAML, not notebooks
#8Reads like someone was forced by their marketing team to write an article about anything at all.
I find the quality of articles on towardsdatascience.com has significantly decreased in the past months. This article is no exception.
Re: Why we do machine learning engineering with YAML, not notebooks
#9The cortex tool mentioned looks really useful to get a service running out of a trained model. Though I didn't really understand what the article is trying to get at. Storing your deployment configuration in yaml and json files is pretty much the standard.
Perhaps for young people it's actually necessary to mention JSON and YAML as you people tend to read the news rather than history or best practises textbooks?
Many hyped development and administration practices make me snarky, too, but let's not take it out on random (young) people on the Internet. There's a great deal of history and best practices that are being ignored not just by ADHD juniors, but also by team leads and managers of every seniority level. Lots of people out there spend decades in the industry, build solid careers based on playing the right office politics cards and being friends with the right people, and miraculously manage not to learn almost anything.
(Edit: also, clearly, I don't read the news -- is YAML being "superseded"? By what now??)
Re: Why we do machine learning engineering with YAML, not notebooks
#10Its quite simple to develop a model, bundle it up for deployment and deploy. Nobody cares about your fancy YAML based containerized deployment and monitoring setup, everyone has that. The challenge comes in when you have a continuous cycle of data ingestion to model optimization, training, evaluation and deployment. Pretty much everybody has huge amount of code duplication in there. It also comes from the fact that ml researchers are barely capable of programming a light switch, like how are you ever gonna put the horrible trash of code they ducked tape together from medium posts into a production environment. Hopeless.