Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your ML stack like?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 134 posts

Re: Ask HN: What is your ML stack like?

#11

Post by brian_spiering »

exhilaration wrote: post #6
aaron-santos wrote: post #5

What didn't work: Shipping pickled models to other teams. Deploying Sagemaker endpoints (too costly). Requiring editing of config files to deploy endpoints. What did work: Shipping http endpoints. Deriving api documentation from model docstrings. Deploying lambdas (less costly than Sagemaker endpoints). Writing a ~150 line python script to pickle the model, save a requirements.txt, some api metadata, and test input/o…

Could you possibly define "pickling" in this context for us ML noobs?

Pickling is a protocol to serialize Python objects. In scikit-learn that would be serializing an Estimator.

https://docs.python.org/3/library/pickle.html https://scikit-learn.org/stable/modules/model_persistence.ht...

Re: Ask HN: What is your ML stack like?

#12

Post by aaron-santos »

exhilaration wrote: post #6
aaron-santos wrote: post #5

What didn't work: Shipping pickled models to other teams. Deploying Sagemaker endpoints (too costly). Requiring editing of config files to deploy endpoints. What did work: Shipping http endpoints. Deriving api documentation from model docstrings. Deploying lambdas (less costly than Sagemaker endpoints). Writing a ~150 line python script to pickle the model, save a requirements.txt, some api metadata, and test input/o…

Could you possibly define "pickling" in this context for us ML noobs?

We save the state of an object (an instance of a class with a predict() method) to disk once we have a model that we are happy with. During deployment we copy this file to a server which loads the file from disk and restores the state of the object on the remote machine.

We use dill[0], but there are other similar libraries.

[0] https://pypi.org/project/dill/

Re: Ask HN: What is your ML stack like?

#13

Post by yubozhao »

aaron-santos wrote: post #5

What didn't work: Shipping pickled models to other teams. Deploying Sagemaker endpoints (too costly). Requiring editing of config files to deploy endpoints. What did work: Shipping http endpoints. Deriving api documentation from model docstrings. Deploying lambdas (less costly than Sagemaker endpoints). Writing a ~150 line python script to pickle the model, save a requirements.txt, some api metadata, and test input/o…

hi Aaron, We did exactly what works for you into a open source python library, github.com/bentoml/bentoml.

It packages your model for you into a standardized format, that you can use it in multiply serving scenarios online serving with api endpoint, offline serving with spark udf, CLI access or import it as python module. It also helps you deploy to different platform such as lambda, sagemaker and others.

Our value is from model in notebook to production service in 5 mins. Love to hear your feedback on this. You can try out our quick start on Google colab (https://colab.research.google.com/github/bentoml/BentoML/blo...)

Re: Ask HN: What is your ML stack like?

#14

Post by PascLeRasc »

exhilaration wrote: post #6
aaron-santos wrote: post #5

What didn't work: Shipping pickled models to other teams. Deploying Sagemaker endpoints (too costly). Requiring editing of config files to deploy endpoints. What did work: Shipping http endpoints. Deriving api documentation from model docstrings. Deploying lambdas (less costly than Sagemaker endpoints). Writing a ~150 line python script to pickle the model, save a requirements.txt, some api metadata, and test input/o…

Could you possibly define "pickling" in this context for us ML noobs?

If by ML noob you mean to say that you're like me and have zero formal CS training (as in, I don't know what a data structure is), pickling lets you write your Python workspace to a file just like Matlab's .mat file loading. It's excellent for writing scripts defining different parts of a data pipeline, or just for debugging/trying new things without waiting 20 minutes for something to filter.

Re: Ask HN: What is your ML stack like?

#15

Post by chronic379019 »

gautamcgoel wrote: post #3

I'm a PhD student at Caltech, working on the theoretical foundations of ML. I personally don't do a lot of coding, but basically everyone in my department uses Python (especially Pytorch) for deep learning/ML. This all runs on Nvidia GPUs (never seen an AMD GPU in the office). Occasionally people code in Matlab, especially if they work in optimization or control. Tmux and git are the only command line tools I see com…

You didn't really describe a stack. Which is fine, because academic research usually doesn't really reuse code ;)

A proper ML stack is something like:

- Data format in X schema

- Model trained on Y library/platform

- Evaluated and tested using Z

- Serialized in A format

- Stored on cloud B

- Deployed using C

- Versioned using D

- Real-time monitoring using E

Re: Ask HN: What is your ML stack like?

#17

Post by ZeroCool2u »

Plotly's Dash to prototype front ends that ingest/use the model output (Our team is Python only, but others use R, so this works great, because it supports both.) https://dash.plot.ly/

FastAPI for quickly creating new API endpoints. It has automatic _interactive_ docs and super simple data validation via Python typehints, so that we don't waste compute time with malformed data. https://fastapi.tiangolo.com/

We deploy on prem most of the time, but have started using GCP on occasion.

Re: Ask HN: What is your ML stack like?

#18

Post by suyash »

For my pet projects, do training and testing locally on my machine either using Notebooks or on an IDE. Test and validate it further on my local machine before deploying it on a server as a micro service. This is for my pet projects only.

Re: Ask HN: What is your ML stack like?

#19

Post by aaron-santos »

yubozhao wrote: post #13
aaron-santos wrote: post #5

What didn't work: Shipping pickled models to other teams. Deploying Sagemaker endpoints (too costly). Requiring editing of config files to deploy endpoints. What did work: Shipping http endpoints. Deriving api documentation from model docstrings. Deploying lambdas (less costly than Sagemaker endpoints). Writing a ~150 line python script to pickle the model, save a requirements.txt, some api metadata, and test input/o…

hi Aaron, We did exactly what works for you into a open source python library, github.com/bentoml/bentoml. It packages your model for you into a standardized format, that you can use it in multiply serving scenarios online serving with api endpoint, offline serving with spark udf, CLI access or import it as python module. It also helps you deploy to different platform such as lambda, sagemaker and others. Our value i…

It's been great seeing this space fill out with solutions in the last year. MLFlow[1] is another open source solution I have my eyes on.

BentoML looks more cohesive than our homegrown solution because it targets a more general case. One of the things I would miss switching to BentoML would be automatic requirements generation. We use pipreqs[2] to generate a requirements.txt given a model instance. Any thoughts on the difficulty as a user in extending BentoML as to integrate pipreqs?

Again another difficulty question: we have a few statsmodels[3] predictors and it isn't clear how much work would be involved extending BentoML to accept those too.

Thanks for pointing out BentoML. I'll keep an eye on it as a migration target as this space develops.

[1] https://mlflow.org/docs/latest/index.html

[2] https://github.com/bndr/pipreqs

[3] https://www.statsmodels.org/stable/index.html

Re: Ask HN: What is your ML stack like?

#20

Post by jacquesm »

Python (the Conda distribution, not small but quite easy and batteries included), Keras, Tensorflow, NVidia hardware (GTX1080ti, not sure what the current sweet spot for price/performance in GPU land is but that was the best I could get at the time).
Post reply on HN