Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

cml.dev

Post reply on HN
Print view View original

11–20 of 40 posts

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#11

Post by rhythmvertigo »

tknaup wrote: post #6

ML is a relatively young field, and decades behind Software Engineering in terms of best practices for running production systems. CI/CD massively improved the innovation cycle time and quality of production software, and I believe it is key for building robust production ML systems as well. CML looks like a really easy to use product for bringing CI/CD to ML projects.

Thanks for the kind words! DevOps has had profound results and we think culturally and technically, it's the right moment to push for those practices in ML.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#12

Post by obowersa »

tknaup wrote: post #6

ML is a relatively young field, and decades behind Software Engineering in terms of best practices for running production systems. CI/CD massively improved the innovation cycle time and quality of production software, and I believe it is key for building robust production ML systems as well. CML looks like a really easy to use product for bringing CI/CD to ML projects.

Just something I wanted to share which might be of interest on this side of things.

The CD Foundation has a SIG around MLOps which is pretty active and has some awesome folk participating.

For anyone who's interested in this space, there's some more detail here: https://cd.foundation/blog/2020/02/11/announcing-the-cd-foun...

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#13

Post by ishcheklein »

Hey! Disclaimer - I'm one of the DVC maintainers :) Super excited for the team on this release!

For the last two years we have seen over and over again how our users take DVC and use it inside Gitlab, Github, etc. This product was born partially as a result of these discussions, partially as an initial visions for the ML tools ecosystem - Hashicorp-like.

Having A software engineering background I really hope that integrating ML workflow into engineering tools will be the future of this space. And with CML and other tools (e.g. https://github.blog/2020-06-17-using-github-actions-for-mlop...) we see this happening.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#15

Post by rhythmvertigo »

ayanb9440 wrote: post #14

Very exciting! I used this team's previous product (DVC) at a research lab at Caltech. This looks like a very useful tool.

Out of curiosity, what kind of research? I'm very interested in getting DVC and Git more broadly into academic research circles, but finding a lot of barriers in my home field.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#16

Post by sytse »

ishcheklein wrote: post #13

Hey! Disclaimer - I'm one of the DVC maintainers :) Super excited for the team on this release! For the last two years we have seen over and over again how our users take DVC and use it inside Gitlab, Github, etc. This product was born partially as a result of these discussions, partially as an initial visions for the ML tools ecosystem - Hashicorp-like. Having A software engineering background I really hope that int…

Very cool to see this!

I've added integrating CML closer into GitLab to our direction page with https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_request...

If you want to show the results in the Merge Request itself like we do for things like test results and security scans please let us know. Open an issue and if there is no response my Twitter handle is @sytses

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#17

Post by rkaplan »

Longtime DVC user here - this is going to be so helpful. We use DVC for all of our model and data versioning, but what's been missing is the ability to cleanly integrate that into our CI workflow. Looks like that's solved now! The cml.yaml syntax also looks quite nice, very easy to follow. Looking forward to trying this out.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#18

Post by bigfoot675 »

I think taking the approach of "help[ing] your team make informed, data-driven decisions" through generating reports is valuable here. In my opinion, it goes too far if we start continuously deploying ML code like it's a SWE project. To take an example in the case of autonomous vehicles, pushing continuous updates to perception modules without thoroughly exploring the ramifications of an update could be potentially catastrophic.

Obviously we can't predict every error by thinking hard, but datasets will never serve as a full representation of what models might experience in the real world. Continuous deployment to an ML model could affect undefined behavior in unpredictable ways.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#19

Post by ishcheklein »

sytse wrote: post #16
ishcheklein wrote: post #13

Hey! Disclaimer - I'm one of the DVC maintainers :) Super excited for the team on this release! For the last two years we have seen over and over again how our users take DVC and use it inside Gitlab, Github, etc. This product was born partially as a result of these discussions, partially as an initial visions for the ML tools ecosystem - Hashicorp-like. Having A software engineering background I really hope that int…

Very cool to see this! I've added integrating CML closer into GitLab to our direction page with https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_request... If you want to show the results in the Merge Request itself like we do for things like test results and security scans please let us know. Open an issue and if there is no response my Twitter handle is @sytses

Hey Sytse! Thanks for the kind words. We're very interested in a deep integration with Gitlab :) Can you share some examples of these security scan reports, please? Right now, we return CML reports as comments in Merge Requests like this: https://gitlab.com/iterative.ai/cml-cloud-case/-/merge_reque.... We'd appreciate any tips or suggestions.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#20

Post by sam-white »

ishcheklein wrote: post #19
sytse wrote: post #16

Earlier quoted context omitted.

Very cool to see this! I've added integrating CML closer into GitLab to our direction page with https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_request... If you want to show the results in the Merge Request itself like we do for things like test results and security scans please let us know. Open an issue and if there is no response my Twitter handle is @sytses

Hey Sytse! Thanks for the kind words. We're very interested in a deep integration with Gitlab :) Can you share some examples of these security scan reports, please? Right now, we return CML reports as comments in Merge Requests like this: https://gitlab.com/iterative.ai/cml-cloud-case/-/merge_reque... . We'd appreciate any tips or suggestions.

For some examples, you can see these designs for including feedback on security scans directly in an MR: https://gitlab.com/gitlab-org/gitlab/-/issues/12896/designs
Post reply on HN