41–50 of 134 posts
why not write the ML model in Java?
why not write the ML model in Java?
why not write the ML model in Java?
The better question, is why Java? I don’t think I’ve ever encountered any company or person to use Java for ML. Scala yes. Clojure, surprisingly, yes. Java, no. Not to say they don’t exist, but it’s not a good idea. The ecosystem isn’t there, and the language (I want to say sucks), isn’t there either.
What didn't work: Shipping pickled models to other teams. Deploying Sagemaker endpoints (too costly). Requiring editing of config files to deploy endpoints. What did work: Shipping http endpoints. Deriving api documentation from model docstrings. Deploying lambdas (less costly than Sagemaker endpoints). Writing a ~150 line python script to pickle the model, save a requirements.txt, some api metadata, and test input/o…
Did you have the two systems talking to each other through HTTP endpoints? I mean the ML system receiving data from a source API and sending back a result? Is this where AWS lambdas jumps in? Are there any formal tools that facilitate making these endpoints?
Currently: Models and feature engineering done in python, trained locally, weights uploaded to S3. Dockerfile with a tiny little web server gets deployed through or CI/CD pipeline for serving. Soon: Argo workflows + Polyaxon for data collection, feature engineering, training etc. Push best model tobS3, same CICD process with docker container deploys little web server onto our Kubernetes environment. Deep learning stu…
Happy to answer any question or provide more information.
[0]: https://github.com/polyaxon/polyaxon
[1]: https://github.com/polyaxon/polyaxon/blob/master/cli/tests/f...
One dead simple way to do this (R model —> Java production) that I’ve done in the past is to use PMML (via pmml package), which converts models to an XML representation. ONNX is a similar/newer framework along these lines. You can also look at dbplyr for performing (dplyr-like) data preprocessing in-database.
why not write the ML model in Java?
The systems I've seen basically break things into different services. Tied together with gRPC or Thrift which have code generators for most languages. So the Java backend simply makes RPC requests to a server running R. Although in one case we had very tight latency requirements (ie: 10ms) so the ML results were pre-computed and loaded from a cache on the backend servers.
I imagine I will be breaking down into different services as well. An ML "blackbox" that makes a call to the back-end for data and returns a result/prediction. This could happen through an API. What kind of API to choose is still open. I'm not very sure what you mean when you say the ML results were pre-computed?
>I'm not very sure what you mean when you say the ML results were pre-computed?
We were scoring ads per page, and possible values for both were known ahead of time. So for each ad-page combination we generated the scores and then pushed them into a giant cache.
PMML is language agnostic model specification (XML like). Python and R machine learning ecosystem can easily generate these (caveat, only tried for gbdt and linear models and not sure this works well for neural nets).
Openscoring is Java library that creates rest API for scoring models. It's lightweight, battle-tested, nice API, good model versioning and in my experience 10x faster than Python flask. You don't need to write any Java code, just download and run the .jar and post valid PMML to the right endpoint.
Another feasible approach is Sagemaker deploy - code from Jupyter notebook can deploy API in one line. I think this can be less economical and have higher latency if you will have high usage but a datascientist can do model updates from within a notebook.
Please NEVER hardcode regression model coefficients within Java. This is a nightmare to maintain, prevents increasing model complexity and is no simpler than PMML + openscoring. I think you can wrap the Java PMML library in another Java web framework like spring if you need something more bespoke.
https://www.rdocumentation.org/packages/pmml/versions/2.1.0/...
https://github.com/openscoring/openscoring
https://aws.amazon.com/blogs/machine-learning/using-r-with-a...
Earlier quoted context omitted.
The better question, is why Java? I don’t think I’ve ever encountered any company or person to use Java for ML. Scala yes. Clojure, surprisingly, yes. Java, no. Not to say they don’t exist, but it’s not a good idea. The ecosystem isn’t there, and the language (I want to say sucks), isn’t there either.
Where did you see clojure being used in production for ML? I am curious, because I am a clojure dev