Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your ML stack like?

news.ycombinator.com

Post reply on HN
Print view View original

61–70 of 134 posts

Re: Ask HN: What is your ML stack like?

#61

Post by sten »

aaron-santos wrote: post #5

What didn't work: Shipping pickled models to other teams. Deploying Sagemaker endpoints (too costly). Requiring editing of config files to deploy endpoints. What did work: Shipping http endpoints. Deriving api documentation from model docstrings. Deploying lambdas (less costly than Sagemaker endpoints). Writing a ~150 line python script to pickle the model, save a requirements.txt, some api metadata, and test input/o…

This is oddly similar to what we're doing. Except my team is heading towards sagemaker against my wishes.

Re: Ask HN: What is your ML stack like?

#62

Post by superkitty »

We use self deployable configuration to allow Data Scientist to control the model's destiny.

Models are written in Python (mix of pytorch/NLP/tensorF). The Models are serving about 35 predictions/second on avg. The API server written in the Python. API server container feeds or write the requests in the distributed queue cluster. The models picks up the samples from the queue in batching. It allows to experiment the models (different flavor) based on the routing being set during the deployment time and which in turns being set in the cache. We use AWS managed cache, queuing and container orchestration platform. Next: 1)Current pipeline for the training and production is two separate pipeline which we want to combined, possibly use MLFlow, Airflow or KubeFlow. Deployment to the production is done through Jenkins. 2)Active retraining and auto deployment to production. 3)Tie the version of model in production to model being trained. There is no way for us to tie back the version.

Re: Ask HN: What is your ML stack like?

#63

Post by imagiko »

oli5679 wrote: post #49

I'd recommend exporting R model as PMML file, and getting your Java team to interact with Openscoring server. PMML is language agnostic model specification (XML like). Python and R machine learning ecosystem can easily generate these (caveat, only tried for gbdt and linear models and not sure this works well for neural nets). Openscoring is Java library that creates rest API for scoring models. It's lightweight, batt…

Looks very interesting. Will definitely explore in this direction.

> Please NEVER hardcode regression model coefficients within Java.

Amen to that.

Re: Ask HN: What is your ML stack like?

#64

Post by FridgeSeal »

superkitty wrote: post #54
FridgeSeal wrote: post #40

Currently: Models and feature engineering done in python, trained locally, weights uploaded to S3. Dockerfile with a tiny little web server gets deployed through or CI/CD pipeline for serving. Soon: Argo workflows + Polyaxon for data collection, feature engineering, training etc. Push best model tobS3, same CICD process with docker container deploys little web server onto our Kubernetes environment. Deep learning stu…

What about the A/B testing? What do you use for A/B strategy. How many predictions are being served by the model per second?

For most of the stuff we’ve deployed, we’re not yet operating at a scale/level of interest where A/B rearing is worth it. Additionally, the purposes we’re using most of these models for don’t really necessitate A/B testing.

When we do need A/B testing, we’ll probably use something like Seldon. As for predictions/second, not very much at the moment: 1 per 30 seconds maybe? It’s not deployed into a Kubernetes cluster because of scaling requirements, it’s because that’s where all our other services greet deployed till, and it’s more beneficial (ops and cost wise) to also deploy into there than it is to bother with having a separate workflow for deploying to lambda’s or SageMaker.

Re: Ask HN: What is your ML stack like?

#65

Post by sten »

rewritteninrust wrote: post #57

I'm also curious—and maybe someone here can chime in—about how you get organizational buy in for introducing ML. There are a couple of problem areas at my company that I think would be great for ML, but I don't know how to get others onboard.

Someone here wrote a POC. They happened to be fairly high up, not C-level but well respected. Basically, I can save x% cost using this. You can bootstrap that if you're not high enough to sit at the table, just need to convince someone who does.

I appreciate that this approach won't work for everyone.

Re: Ask HN: What is your ML stack like?

#66

Post by lev272 »

derision wrote: post #59
aaron-santos wrote: post #5

What didn't work: Shipping pickled models to other teams. Deploying Sagemaker endpoints (too costly). Requiring editing of config files to deploy endpoints. What did work: Shipping http endpoints. Deriving api documentation from model docstrings. Deploying lambdas (less costly than Sagemaker endpoints). Writing a ~150 line python script to pickle the model, save a requirements.txt, some api metadata, and test input/o…

Would love to hear your thoughts on this? cortex.dev

We use Cortex and I'd say I'm pleased with it. It doesn't offer the end-to-end solution that something like SageMaker does, but it's the best tool we've used for deploying models. Also, and this is less of a technical feature and more of a nice to have, but the team has been really responsive when we've had problems and they seem to be shipping new features at a steady clip.

Re: Ask HN: What is your ML stack like?

#67

Post by FridgeSeal »

superkitty wrote: post #55
mmq wrote: post #45

Earlier quoted context omitted.

Glad to see that you are interested by using Polyaxon[0] for your MLOps. Although I was going to write a blog post about the upcoming v1.0 release of Polyaxon, I just wanted to point out that there will be a native support for different type of workflows, currently it supports parallelism and distributed learning, and in the next release there will be native support for DAGs as well. Here's a test fixture[1] of what…

Any inputs on Argo workflows vs Kubeflow vs MLFlow? Which is better suited?

Kubeflow, unless I’m missing some things, is for Tensorflow pipelines, if you’re not using TF, or it’s not the only thing you use, it’s not ideal.

I thought MlFlow was a spark thing, and were trying to migrate off of spark/DataBricks due to the resources inefficiencies of Spark (at our scale) and maintenance nightmare that python notebooks are causing us.

Argo is just a container workflow tool, not ML specific. We’re planning on using Argo for the data engineering parts, and polyaxon for the ML training parts because of the convenient monitoring and hyper parameter search tools.

Re: Ask HN: What is your ML stack like?

#68

Post by nravic »

rozgo wrote: post #60

Custom Unreal Engine simulator, simulating agents with NVidia Physx and publishing sensors through GStreamer. GStreamer has sinks and sources for ROS, and tensorflow elements for inferencing. We package this all into NVidia Docker for scalable simulations. Setup is similar for training and inference. The core framework is a streaming engine with stream combinators that enable reasoning about spatio-temporal data stre…

This is really cool, what are you training?

Re: Ask HN: What is your ML stack like?

#70

Post by eigenvalue »

rozgo wrote: post #60

Custom Unreal Engine simulator, simulating agents with NVidia Physx and publishing sensors through GStreamer. GStreamer has sinks and sources for ROS, and tensorflow elements for inferencing. We package this all into NVidia Docker for scalable simulations. Setup is similar for training and inference. The core framework is a streaming engine with stream combinators that enable reasoning about spatio-temporal data stre…

How well does the training in the simulator transfer over to real world environments? Do the models require fine tuning on real life data, or do they immediately work? I would imagine they would get confused by things like wheel/tracks slipping a bit on the ground versus in the Unreal simulation.
Post reply on HN