Snorkel AI: Putting Data First in ML Development
1–10 of 67 posts
Re: Snorkel AI: Putting Data First in ML Development
#2Forgive me - but how does this avoid the chicken&egg problem here. Without digging through the promo copy, why would one programmatically label training data to do ML on if they have such a program to label data...
Re: Snorkel AI: Putting Data First in ML Development
#3Forgive me - but how does this avoid the chicken&egg problem here. Without digging through the promo copy, why would one programmatically label training data to do ML on if they have such a program to label data...
That's a really good question. I took a class with one of the professors who started Snorkel.
The way he broke it down was either you can incorporate rules into your data or into your model. Because we want our model to be as general purpose as possible, it turns out you can squeeze some extra performance by "bronze/copper" quality data with handwritten rules in your dataset.
You can think of the model getting an extra boost from the latent knowledge within the rules.
Re: Snorkel AI: Putting Data First in ML Development
#4They totally ignored the open source package in their site.
https://github.com/snorkel-team/snorkel
Moreover, based on the commit history in github, and their README.md , my understanding is that they will stop supporting their open source repo. Best of luck for them.
Re: Snorkel AI: Putting Data First in ML Development
#5Forgive me - but how does this avoid the chicken&egg problem here. Without digging through the promo copy, why would one programmatically label training data to do ML on if they have such a program to label data...
Labels are knowledge about data. If you already know some rules that work reasonably well based on your domain experience, then Snorkel lets you capture those as "labeling functions" that may not cover the whole ground or can be "noisy". Snorkel can then build a model to label your data accounting for the "noise". Combining that with some "gold" labels (done by humans), you can use the generated labels on a large data set to build a higher quality model that generalizes better. This is similar to how you can take several low quality models and by virtue of them having expertise over different parts of the data, build an "ensemble" model that performs better than any of them.
Imho, Snorkel kinds of tools ("weak supervision") are game changers for ML .. though the biggies get all the press. So I'm excited to see this end to end direction taken by the team.
Re: Snorkel AI: Putting Data First in ML Development
#6Looks like they selling snake oil on the name of Stanford and AI :)
Re: Snorkel AI: Putting Data First in ML Development
#7Forgive me - but how does this avoid the chicken&egg problem here. Without digging through the promo copy, why would one programmatically label training data to do ML on if they have such a program to label data...
Labels are knowledge about data. If you already know some rules that work reasonably well based on your domain experience, then Snorkel lets you capture those as "labeling functions" that may not cover the whole ground or can be "noisy". Snorkel can then build a model to label your data accounting for the "noise". Combining that with some "gold" labels (done by humans), you can use the generated labels on a large dat…
is not this done for years and called synthetic data generation, simulation etc.
Re: Snorkel AI: Putting Data First in ML Development
#8Does anyone use Snorkel (https://github.com/snorkel-team/snorkel)? From what I can tell, it seems like a research product. Not sure if any company uses them in a production environment.
Re: Snorkel AI: Putting Data First in ML Development
#9Forgive me - but how does this avoid the chicken&egg problem here. Without digging through the promo copy, why would one programmatically label training data to do ML on if they have such a program to label data...
I saw a talk on Snorkel a few years back, so I don't remember perfectly, but it seemed to be an iterative process. It's a tool for you to build and refine simple rules. If you have ingredients, a simple heuristic " " will get a lot of them, but there are tons of edge cases. With more heuristics, you might get lots of those, and so. I think it was a tool to help you explore and iterate on those heuristic labeling functions quickly. Then you can label the stuff that's hard in a more expensive way or something. I thought of it as noisily hand labeling sets of examples at a time rather than single examples at a time. This is all memory from a random conference talk or paper or something years ago so take it with some big grains of salt. I do clearly remember thinking it seemed really cool at the time.
Re: Snorkel AI: Putting Data First in ML Development
#10Forgive me - but how does this avoid the chicken&egg problem here. Without digging through the promo copy, why would one programmatically label training data to do ML on if they have such a program to label data...
Their paper explains it - https://arxiv.org/abs/1711.10160
Snorkel itself has been a open source package for a while - https://github.com/snorkel-team/snorkel
This new announcement is about Snorkel Flow