91–100 of 134 posts
crystal / shainet ( https://github.com/NeuraLegion/shainet ) I contract for some clients in fintech and some defense-related stuff.
What kind of targets is crystal used on? Strictly x86_64 linux? EDIT: Took a little digging but I found it [1] (and yes it's primarily x86_64 linux+macOS) [1] https://github.com/crystal-lang/crystal/wiki/Platform-Suppor...
Earlier quoted context omitted.
Glad to see that you are interested by using Polyaxon[0] for your MLOps. Although I was going to write a blog post about the upcoming v1.0 release of Polyaxon, I just wanted to point out that there will be a native support for different type of workflows, currently it supports parallelism and distributed learning, and in the next release there will be native support for DAGs as well. Here's a test fixture[1] of what…
This looks increadibly exciting. What's the story around what happens after training/validation? I can't see anything on your site specific to this - do you currently (or plan to) offer anything to help track or version model "releases"?
From Polyaxon's side, we will be providing a set of reusable components, some of these components will be targeting deployment and packaging, for instance aws lambda, azure ml, sagemaker, open source projects for deployment (some of them are mentioned in this HN thread) Users can also contribute components as well or create them to be used inside there organization.
For versioning, all component have versions, and all runs have full overview of their dependencies and provenance (inputs/outputs). This gives all information about when and how a model was created.
The platform knows how to create and manage services, e.g. tensorboards, notebooks, dash, simulators for RL agents ..., so one can also deploy the model as an internal Polyaxon service to be used as an internal tool or for testing purposes, the only thing to keep in mind is that the API endpoint will be subject to the same access rights as other components create by a given user.
Earlier quoted context omitted.
If by ML noob you mean to say that you're like me and have zero formal CS training (as in, I don't know what a data structure is), pickling lets you write your Python workspace to a file just like Matlab's .mat file loading. It's excellent for writing scripts defining different parts of a data pipeline, or just for debugging/trying new things without waiting 20 minutes for something to filter.
> as in, I don't know what a data structure is Basically everything you work with in programming is a value (the number one-hundred-seventy-five, for example: "175") or the address—location in computer memory, say—of a value. You might record the address of that value above as the count of characters from the beginning of this post, for example, were this post the layout of data in some RAM, just as numbering houses…
Any other clear descriptions of CS concepts - especially for Data Science - to share ? Links to them ?
1. Use R packages to bundle up the models with a consistent interface.
2. Create thin Plumber APIs that wrap these packages/models.
3. Build Docker images from these APIs.
4. Deploy API containers to a Docker Swarm (but you could use any orchestration).
5. Stick Nginx in front of them to get pretty, human-readable routes.
6. Call the models via HTTP from C#.
This stack works pretty well for us. Response times are generally fast and throughput is acceptable. And the speed at which we can get models into production is massively better than when we used to semi-manually translate model code to C#...
Probably the biggest issue was getting everyone on board with a standard process and API design, after a few iterations. And putting in place all the automation/process/culture to help data teams write robust, production-ready software.
We don't yet have a polyglot architecture, but we do have the requirement of running distributed services (partly because there are certain components of the pipeline that needs to be run on-premise), and we have found that workflow engines / orchestrators definitely makes it a lot easier to reason about the wider architecture / have a bird's eye view. It works for us. No need to handle callbacks, events, queues, etc. We also do have the potential to run a polyglot architecture.
We tried out Celery Workflows, and struggled to get it "production ready", so I'd advise against this for complex workflows. We also found the visibility lacking.
We have yet to fully try out Kubeflow, and MLflow. What is not quite working at the moment is creating, and deploying portable models. And I don't mean simply pickling, and storing an artifact.
Leveraging containers (Docker), and slapping simple anti-corruption layers (e.g. simple web APIs) has also helped. We have a more consistent way of deploying, and isolating code without having to rewrite much.
We want to look into using Nuclio, and/or knative to ease the process of deployment, and to empower the data scientists to deliver without much engineering expertise.
Others have mentioned using base classes or standard interfaces for their models. We tried this too, but it didn't work. The generalisation early on was met with conflicting requirements, and broke the interface segregation principle (not that it matters too much, but it can be confusing to not know precisely what is being used or not used). We figured it's much easier to procrastinate any abstractions. Let the data, and it's flow do the talking.
Earlier quoted context omitted.
> as in, I don't know what a data structure is Basically everything you work with in programming is a value (the number one-hundred-seventy-five, for example: "175") or the address—location in computer memory, say—of a value. You might record the address of that value above as the count of characters from the beginning of this post, for example, were this post the layout of data in some RAM, just as numbering houses…
I have never read such a crystal clear explanation of what a computer / a Von Neumann architecture is. Simpler is always better. Thanks a lot for this. Any other clear descriptions of CS concepts - especially for Data Science - to share ? Links to them ?
Harness exposes a framework for adding Engines and does all the routing for Engine Instance workflow and lifecycle management. It also provides a toolbox of abstractions for using the Spark ecosystem with Mongo and Elasticsearch.
It comes in a docker-compose system for vertical scaling and Kubernetes for ultimate in scaling an automation. Quite a nice general system with out of the box usefulness.
Earlier quoted context omitted.
Hey Aaron, I work on Cortex which is a tool for continuously deploying models as HTTP endpoints on AWS. Under the hood we use Kubernetes instead of Lambda to avoid cold starts, enable more flexibility with customizing compute and memory usage (e.g. running inference on GPUs), and support spot instances. Could you clarify your comment regarding editing of config files? Is it still a problem if the configuration is dec…
Sure, I'm thinking about the development lifecycle in terms of what actions data scientists have to take to get a model deployed. Anytime the process has a branch (ie: you need to change this file whenever something elsewhere changes) then I know I'm going to forget to do that. If we were to use Cortex, we would likely wrap the creation of cortex.yml in a function and call it when we're saving our models. We do somet…
Custom Unreal Engine simulator, simulating agents with NVidia Physx and publishing sensors through GStreamer. GStreamer has sinks and sources for ROS, and tensorflow elements for inferencing. We package this all into NVidia Docker for scalable simulations. Setup is similar for training and inference. The core framework is a streaming engine with stream combinators that enable reasoning about spatio-temporal data stre…