Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

cml.dev

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#21

Post by rhythmvertigo »

bigfoot675 wrote: post #18

I think taking the approach of "help[ing] your team make informed, data-driven decisions" through generating reports is valuable here. In my opinion, it goes too far if we start continuously deploying ML code like it's a SWE project. To take an example in the case of autonomous vehicles, pushing continuous updates to perception modules without thoroughly exploring the ramifications of an update could be potentially c…

Definitely- automatic model deployment would only work for a subset of applications. We think a lot more about having models available as candidates for deployment, which can then be inspected by domain experts on a team.

One of our motivations for building visual reports that appear like comments in a pull request is giving teams metrics & info to discuss when deciding if merge is right. That way, the automated part is training and testing, but the decision making is human (i.e., data scientists whose skills are better used interpreting models & data than running repetitive training scripts).

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#22

Post by rhythmvertigo »

rkaplan wrote: post #17

Longtime DVC user here - this is going to be so helpful. We use DVC for all of our model and data versioning, but what's been missing is the ability to cleanly integrate that into our CI workflow. Looks like that's solved now! The cml.yaml syntax also looks quite nice, very easy to follow. Looking forward to trying this out.

Really glad to hear that, rkaplan. Let us know if we can be of any help!

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#23

Post by ishcheklein »

bigfoot675 wrote: post #18

I think taking the approach of "help[ing] your team make informed, data-driven decisions" through generating reports is valuable here. In my opinion, it goes too far if we start continuously deploying ML code like it's a SWE project. To take an example in the case of autonomous vehicles, pushing continuous updates to perception modules without thoroughly exploring the ramifications of an update could be potentially c…

Agreed! I think, it's similar to software engineering. It's not that often that large websites are being deployed to prod without someone approving/QA-ing it manually first, right?

CI/CD systems in this case help automating this as much as possible, but do not completely replace decision making process, I would say.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#25

Post by rhythmvertigo »

m0sth8 wrote: post #24

CML looks really awesome. I've been on one of your online meetups. Are you planning to host more in the future? It would be great to learn real production use cases!

Yes, we're aiming to have another at the end of July! If you have a particular use case you're interested in, we'd love to know. We might be able to develop some materials around it.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#26

Post by stkerr »

ishcheklein wrote: post #19
sytse wrote: post #16

Earlier quoted context omitted.

Very cool to see this! I've added integrating CML closer into GitLab to our direction page with https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_request... If you want to show the results in the Merge Request itself like we do for things like test results and security scans please let us know. Open an issue and if there is no response my Twitter handle is @sytses

Hey Sytse! Thanks for the kind words. We're very interested in a deep integration with Gitlab :) Can you share some examples of these security scan reports, please? Right now, we return CML reports as comments in Merge Requests like this: https://gitlab.com/iterative.ai/cml-cloud-case/-/merge_reque... . We'd appreciate any tips or suggestions.

Our public documentation on these reports and how you can integrate with them is at https://docs.gitlab.com/ee/development/integrations/secure/ in our documentation if you want to check it out! tl;dr - we have a JSON format for security results that if you format your results in, it will show up in the various interfaces in the product automatically.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#27

Post by sytse »

ishcheklein wrote: post #19
sytse wrote: post #16

Earlier quoted context omitted.

Very cool to see this! I've added integrating CML closer into GitLab to our direction page with https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_request... If you want to show the results in the Merge Request itself like we do for things like test results and security scans please let us know. Open an issue and if there is no response my Twitter handle is @sytses

Hey Sytse! Thanks for the kind words. We're very interested in a deep integration with Gitlab :) Can you share some examples of these security scan reports, please? Right now, we return CML reports as comments in Merge Requests like this: https://gitlab.com/iterative.ai/cml-cloud-case/-/merge_reque... . We'd appreciate any tips or suggestions.

Awesome! Other replies to your comment already gave suggestions. Consider asking them for a call to give in if you need more information.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#28

Post by ishcheklein »

sytse wrote: post #27
ishcheklein wrote: post #19

Earlier quoted context omitted.

Hey Sytse! Thanks for the kind words. We're very interested in a deep integration with Gitlab :) Can you share some examples of these security scan reports, please? Right now, we return CML reports as comments in Merge Requests like this: https://gitlab.com/iterative.ai/cml-cloud-case/-/merge_reque... . We'd appreciate any tips or suggestions.

Awesome! Other replies to your comment already gave suggestions. Consider asking them for a call to give in if you need more information.

Thanks! we'll check it out and be in touch.

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#29

Post by m0sth8 »

m0sth8 wrote: post #24

CML looks really awesome. I've been on one of your online meetups. Are you planning to host more in the future? It would be great to learn real production use cases!

Btw. Here's a recording of the first DVC meetup https://www.youtube.com/watch?v=19GMtrFykSU I like it a lot. Highly recommended!

Re: Show HN: Continuous Machine Learning – CI/CD for Machine Learning Projects

#30

Post by gravypod »

rhythmvertigo wrote: post #2

Hi, I'm one of the project creators. Continuous Machine Learning (CML) is an open source project to help ML projects use CI/CD with Github Actions and Gitlab CI ( https://github.com/iterative/cml ). CML automatically generates human-readable reports with metrics and data viz in every pull/merge request, and helps you use storage and GPU/CPU resources from cloud services. CML addresses three hurdles for making ML comp…

This looks really amazing. Do you have any plans to support on-prem GPUs? If we had Gitlab runners with GPUs in them would this project pick it up and use those runners for training/analysis?
Post reply on HN