Looks like they selling snake oil on the name of Stanford and AI :)
Why do think that? It's a very powerful and useful technique. You can get supervised labels in a day (ostensibly for free) vs paying humans to do it and waiting...
Snorkel AI: Putting Data First in ML Development
31–40 of 67 posts
Re: Snorkel AI: Putting Data First in ML Development
#32As another option, Compose is a ML tool for labeling data. It structures the labeling process and integrates easily with Featuretools which automates feature engineering. Definitely worth checking out!
[1] https://github.com/FeatureLabs/compose [2] https://github.com/FeatureLabs/featuretools
Re: Snorkel AI: Putting Data First in ML Development
#33Myself and a few colleagues attended the _absolutely packed_ presentation the Snorkel folks did at ODSC in Boston last year and came away so convinced by this approach that we actually constructed our interns summer project around using Snorkel for multi-label classification on complex very domain specific financial documents. The project was successful and both us and our intern were very happy.
The main lessons learned from this exercise helped us identify where our efforts would be shifted when using Snorkel. Of course there's never any free lunch, but Snorkel has what I believe to be a very reasonable and effective trade off. Snorkel provides both a huge decrease in overall costs, but critically it shifts costs towards the front of the development process. Writing a good set of labeling functions is a non-trivial piece of work. It requires the data scientist to have deep domain experience or a few fairly large blocks of time collaborating with and learning from a business user that is a already a domain expert. It has the upside though of forcing the data scientist to get a solid foundation of this domain knowledge, which I feel often times is underestimated in many ML projects.
Anyways, congrats to the team! Looking forward to checking out your future work.
Re: Snorkel AI: Putting Data First in ML Development
#34Myself and a few colleagues attended the _absolutely packed_ presentation the Snorkel folks did at ODSC in Boston last year and came away so convinced by this approach that we actually constructed our interns summer project around using Snorkel for multi-label classification on complex very domain specific financial documents. The project was successful and both us and our intern were very happy. The main lessons lea…
can you educate me on how you do multi-label w/ snorkel? As far as I can understand that's one it's largest drawbacks.
Re: Snorkel AI: Putting Data First in ML Development
#35They totally ignored the open source package in their site. https://github.com/snorkel-team/snorkel Moreover, based on the commit history in github, and their README.md , my understanding is that they will stop supporting their open source repo. Best of luck for them.
If anyone is looking for an open source library in this space, I work on one called Compose ( https://github.com/FeatureLabs/compose ). With compose, a user defines a labeling function, and then compose scans the historical data looking for training examples to train a machine learning model. The library has evolved as we apply it to more and more real world use cases, but it is based on approach in this paper from 2…
can you do multi-label w/ Compose? Snorkel only supports single-label.
Re: Snorkel AI: Putting Data First in ML Development
#36As another option, Compose is a ML tool for labeling data. It structures the labeling process and integrates easily with Featuretools which automates feature engineering. Definitely worth checking out! [1] https://github.com/FeatureLabs/compose [2] https://github.com/FeatureLabs/featuretools
can you do multi-label w/ Compose? Snorkel only supports single-label.
Re: Snorkel AI: Putting Data First in ML Development
#37for those who want more on Snorkel, I recommend checking out Chris Re's prior talks:
- https://www.youtube.com/watch?v=yu15Nf5eJEE (14 min)
- https://www.cs.ucla.edu/upcoming-events/cs-201-jon-postel-di... (1hr talk - snorkel's predecessor was deepdive)
I went down the rabbit hole in this space about 3-4 years ago, and really got the message around "Dark Data". he's onto something huge and I regret not pursuing it further due to self doubt. hedge funds should be eating this up as well.
Re: Snorkel AI: Putting Data First in ML Development
#38I presume this does not apply to computer vision datasets? Frankly I am still confused at what exactly Snorkel does.
Re: Snorkel AI: Putting Data First in ML Development
#39I can see how this would work for tabular and text data, where the labeling functions are well-defined. I don't understand, at all, how this would work with computer vision tasks where heuristics are pretty much impossible to define.
That said, I don't see anything here that would prevent you from using a pre-trained conv net as a labeling function, but I expect that multiple conv nets trained on a small corpus of data would be biased and make correlated errors, which violate their assumptions.
This looks super powerful in some cases, but I'm just not seeing how it can possibly generalize to every ML problem.
Re: Snorkel AI: Putting Data First in ML Development
#40I don't understand why this approach is valuable. If you can label your examples using traditional software, why would you build a model? Why not just use the label function in whatever context you intended the model to exist?