Live data from Hacker News

CometML wants to do for machine learning what GitHub did for code

techcrunch.com

11–20 of 65 posts

Re: CometML wants to do for machine learning what GitHub did for code

#12
post #8

Can you talk about how CometML fits into the real world state of ML training tracking, which is a pretty terrible Wild West of non-reproducible practices and processes? There have been articles and comments here on HN about the sorry state of ML trackability, with papers being published on models whose training is not reproducible because no one really knows how it was trained. One in particular (I apologize for not…

That's a great point! Machine Learning reproducibility is a huge problem, both in Academia and within companies. Comet.ml tracks every run of your script, the hyper-params (pulled automatically from the ML library when possible) and your results. So in the example you gave we would be able to track the DS loading pre-trained weights, manually changing hyperparams and the datasets hash.

In such complex cases there's still a little discipline required on the user behalf but overall it's just including a few calls to our Experiment object.

Re: CometML wants to do for machine learning what GitHub did for code

#13
post #7

Great to see ML "governance" work being done on the training-part of the pipeline. Seems like this provides a Domino Data Labs based dashboards but without the walled garden environment. I've yet to see similar great initiatives also tackling the deployment-part. E.g. something similar you can stick on top of your model's API (or scheduled batch predictive outputs), as well as incoming instances, to monitor usage pat…

Thanks! We indeed solve a similar pain point as Domino but we unlike them we allow you to train your models on your own infra/laptop.

As for monitoring production models that's something we're also working on. It was important to get the training part out first so we can measure those distributions changing.

Re: CometML wants to do for machine learning what GitHub did for code

#15

I have been anxiously waiting for something like this! Very excited to try it out. How does it handle data storage? Could we use CometML to store our continuously growing set of labeled data or is there a smart way to link it to Google Cloud Platform?

We do not store your data as it's usually a very sensitive. You can host your data on GCloud or AWS and use Comet.ml to track where it was coming from and if it changed between experiments.

Re: CometML wants to do for machine learning what GitHub did for code

#16
post #2

Hi, I’m one of the founders of Comet.ml. We built comet.ml to allow machine learning teams to automatically track their machine learning code, experiments, hyperparameters and results. We think that reproducibility is really important so we’re also giving free access to students, academics and open source projects. Feedback is welcome. Ask me anything.

Looks awesome. I'd love to give it a try with my team. Would you be open to extending the free teams plan to high tech nonprofits in addition to the access you provide to students, academics, and open source projects? We tend to work on distributed projects frequently with industry experts doing pro bono work for us, and something like comet could simplify our collaboration. The size of these pro bono project teams tend to ebb and flow, much like an open source project, so effective collaboration tools are critical for ramping up new folks as well as retaining learning when folks cycle off.

Re: CometML wants to do for machine learning what GitHub did for code

#18
post #16
post #2

Hi, I’m one of the founders of Comet.ml. We built comet.ml to allow machine learning teams to automatically track their machine learning code, experiments, hyperparameters and results. We think that reproducibility is really important so we’re also giving free access to students, academics and open source projects. Feedback is welcome. Ask me anything.

Looks awesome. I'd love to give it a try with my team. Would you be open to extending the free teams plan to high tech nonprofits in addition to the access you provide to students, academics, and open source projects? We tend to work on distributed projects frequently with industry experts doing pro bono work for us, and something like comet could simplify our collaboration. The size of these pro bono project teams t…

Sure. Shoot us an email and we'll get you started! mail@comet.ml

Re: CometML wants to do for machine learning what GitHub did for code

#19
post #2

Hi, I’m one of the founders of Comet.ml. We built comet.ml to allow machine learning teams to automatically track their machine learning code, experiments, hyperparameters and results. We think that reproducibility is really important so we’re also giving free access to students, academics and open source projects. Feedback is welcome. Ask me anything.

Hi, looks great! Love the feature of tracking code changes. But how does this work? Does it upload the code to the servers every time I launch a training?

Re: CometML wants to do for machine learning what GitHub did for code

#20
post #17

I would like to use it but I think the price doesn't justify the tool. For a team of 5 people github is $25 a month, you are $745 a month. I do understand a price a bit higher that github but not 30 times more expensive.

Thanks @jorgemf. Keep in mind that $745 also includes unlimited usage of our hyper-parameter optimization service.
Post reply on HN