Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning in Production (CMU Course)

mlip-cmu.github.io

Post reply on HN
Print view View original

31–39 of 39 posts

Re: Machine Learning in Production (CMU Course)

#31

Post by ggddv »

underdeserver wrote: post #29
Babawomba wrote: post #27

The material is definitely practical—Kafka, Docker, Kubernetes, and Jenkins are all industry-standard tools, and the focus on MLOps is refreshing. It’s great to see a course bridge the gap between ML and actual production systems, not just stop at building models. Love that they're also tackling explainability, fairness, and monitoring. These are the things that often get overlooked in practice. Is it too entry-level…

Too entry level? Even if every tool is entry level, tying them all together and actually making it work is hard. I'd say it's mid-to-late B.Sc. material. Relevance? Is there really a huge conceptual difference between Jenkins and the other CI/CD frameworks? If not, if I were them I would just choose a random popular one, and it seems to me that's just what they did.

It’s kind of funny all those supposedly complicated technologies are actually pretty simple when you understand why you are using them. Docker is the best example, it’s hard to understand what is happening unless to understand the problem it’s solving.

Re: Machine Learning in Production (CMU Course)

#32

Post by szvsw »

belter wrote: post #30

This seems to have very little on Data Quality and it is on Chapter 16...How much practical experience in Industry do the authors have? Because 90% of your time will be spent on Data Quality and Data Cleansing...

Arguably that’s a separate (obviously critical) concern. I think it’s worth it to abstract that away as just a step that exists in the pipeline with its own set of concerns/challenges/methods etc that really requires its own deeper study to do well.

For instance, my ML work is almost entirely in the context of engineering simulation regression/surrogate development, where data quality/cleaning is almost no issue at all - all of the work is on the dataset generation side and on the model selection/training/deployment side.

Every job is different!

Re: Machine Learning in Production (CMU Course)

#33

Post by kkukshtel »

Babawomba wrote: post #27

The material is definitely practical—Kafka, Docker, Kubernetes, and Jenkins are all industry-standard tools, and the focus on MLOps is refreshing. It’s great to see a course bridge the gap between ML and actual production systems, not just stop at building models. Love that they're also tackling explainability, fairness, and monitoring. These are the things that often get overlooked in practice. Is it too entry-level…

I think what you're missing here is that this is now _the_ entry point for year 1 CS students. People come in wanting to do ML. 20 years ago people came in and learned to write databases with Java and used similarly "will probably be deprecated tools". This is just the new starting point.

Re: Machine Learning in Production (CMU Course)

#34

Post by golly_ned »

SnorkelTan wrote: post #11
golly_ned wrote: post #5

I've worked on ML platforms and systems for 9.5 years at every scale. The material looks great.

What would you recommend for a backend developer looking to make the switch from rest crud apps to ml platforms?

It depends on your starting point. A baseline level of ML is needed. Otherwise ML platforms account for three basic functions: features/data, model training, and model hosting.

So do an end-to-end project where you:

- start from a CSV dataset, with the goal of predicting some output column. A classic example is predicting whether a household's income is >$50K or not from census information.

- transform/clean the data in a jupyter notebook and engineer features for input into a model. Export the features to disk into a format suitable for training.

- train a simple linear model using a chosen framework: a regressor if you're predicting a numerical field, a classifier if its categorical.

- iterate on model evaluation metrics through more feature engineering, scoring the model on unseen data to see its actual performance.

- export the model in such a way it can be loaded or hosted. The format largely depends on the framework.

- construct a docker container that exposes the model over HTTP and a handler for receiving prediction requests and transforming them for input into the model, and a client that sends requests to that model.

That'll basically get an entire end-to-end run the entire MLE lifecycle. Every other part of development is a series of concentric loop between these steps, scaled out to ridiculous scale in several dimensions: number of features, size of dataset, steps in a data/feature processing pipeline to generate training datasets, model architecture and hyperparameters, latency/availability requirements for model servers...

For bonus points:

- track metrics and artifacts using a local mlflow deployment.

- compare performance for different models.

- examine feature importance to remove unnecessary (or net-negative) features.

- use a NN model and train on GPU. Use profiling tools (depends on the framework) and Nvidia NSight to examine performance. Optimize.

- host a big model on GPU. Profile and optimize.

IMO: the biggest missing piece for ML systems/platform engineers is how to feed GPUs. If you can right-size workloads and feed a GPU with MLE workloads you'll get hired. MLE workloads vary wildly (ratio of data volume in vs. compute; size of model; balancing CPU compute for feature processing with GPU compute for model training). We're all working under massive GPU scarcity.

Re: Machine Learning in Production (CMU Course)

#35

Post by bigmutant »

belter wrote: post #30

This seems to have very little on Data Quality and it is on Chapter 16...How much practical experience in Industry do the authors have? Because 90% of your time will be spent on Data Quality and Data Cleansing...

Agree, Data Quality in-the-wild is a huge concern. I've led efforts to establish Lineage/Quality in large orgs and doing this after-the-fact is a massive undertaking. Having this "up-front" before all the data pipelines (origination, transformation, pre-processing) calcify saves a lot of headache down the road.

Re: Machine Learning in Production (CMU Course)

#36

Post by zi_ »

golly_ned wrote: post #34
SnorkelTan wrote: post #11

Earlier quoted context omitted.

What would you recommend for a backend developer looking to make the switch from rest crud apps to ml platforms?

It depends on your starting point. A baseline level of ML is needed. Otherwise ML platforms account for three basic functions: features/data, model training, and model hosting. So do an end-to-end project where you: - start from a CSV dataset, with the goal of predicting some output column. A classic example is predicting whether a household's income is >$50K or not from census information. - transform/clean the data…

this is unbelievably helpful, wow -- thank you!

curious: which part of the pipeline does the majority of 'business' value come from?

Re: Machine Learning in Production (CMU Course)

#37

Post by jononor »

zi_ wrote: post #36
golly_ned wrote: post #34

Earlier quoted context omitted.

It depends on your starting point. A baseline level of ML is needed. Otherwise ML platforms account for three basic functions: features/data, model training, and model hosting. So do an end-to-end project where you: - start from a CSV dataset, with the goal of predicting some output column. A classic example is predicting whether a household's income is >$50K or not from census information. - transform/clean the data…

this is unbelievably helpful, wow -- thank you! curious: which part of the pipeline does the majority of 'business' value come from?

For the majority of usecases I have seen: solving a sufficiently large painpoint, understanding/formulating the problem, having/getting the right data, fitting well into a workflow of the users.

All the technology challenges are actually on the "cost" side of the equation. Meaning, that the aim wrt business value should be do as little of it as possible (but not less!). For some use cases this can still be quite a lot... But more often on the "all the pieces need to be in place for the whole to work at all" rather than "each piece needs to be super optimized".

Re: Machine Learning in Production (CMU Course)

#39

Post by pl4nty »

golly_ned wrote: post #34
SnorkelTan wrote: post #11

Earlier quoted context omitted.

What would you recommend for a backend developer looking to make the switch from rest crud apps to ml platforms?

It depends on your starting point. A baseline level of ML is needed. Otherwise ML platforms account for three basic functions: features/data, model training, and model hosting. So do an end-to-end project where you: - start from a CSV dataset, with the goal of predicting some output column. A classic example is predicting whether a household's income is >$50K or not from census information. - transform/clean the data…

this is really helpful, thanks. how much are third-party models changing these workflows (LLMs etc)? would you still spend as much time on feature engineering and evaluation? I'm wondering whether any saved time would be refocused on hosting, especially optimizing GPU utilization
Post reply on HN