Somewhat unrelated: Deepmind's website is so cluttered and distracting to the extent that it is almost unusable
For all the talk on how clients and managers tend to bikeshed...
Open Source Datasets
11–20 of 29 posts
Re: Open Source Datasets
#12Somewhat unrelated: Deepmind's website is so cluttered and distracting to the extent that it is almost unusable
reminds me of the motherfucking website! http://motherfuckingwebsite.com/
Re: Open Source Datasets
#13Dear Deepmind, as you have retired AlphaGo please open source the dataset of Go games used to train it.
Re: Open Source Datasets
#14I have been thinking about what this solves in respect to other datasets. Nearly all shape recognition datasets have a restriction that you can't use unless you are an academic. I feel like that Open Sourcing data sets will allow us to be more democratic with data and the things that are generated by them. Creative Commons seems like a good license for this though. Once you have the data is half the battle . The rest…
> Creative Commons seems like a good license for this though
I don't think that CC licenses are a good fit for data collections in general. Let's see what other open data projects do:
Some open data projects were not satisfied with CC licenses, which is why ODbL was created:
https://opendatacommons.org/licenses/odbl/
Also, many open data projects choose to put their stuff into the public domain (or license it as "CC0" which means exactly the same). This is, for example, what Wikidata does:
https://www.wikidata.org/wiki/Wikidata:Main_Page
(Wikidata the structured sister project of Wikipedia.)
Re: Open Source Datasets
#15One question that is not clear to me is what should the dataset license to allow / restrict, in the perfect world. For me (just a personal opinion) it would allow free (as in liberty) use, but somehow encourage those who use it to share the benefits (data, software or algorithms) under the same license.
Unfortunately, Open Source does not help here -- I do not see how OS can be used with data sets. The main OS leverage with software development is that if you use software X to build software Y, X is usually present in some way, shape or form in your deliverable Y. Not so with training data -- once algorithm development is done you can (and usually do) strip training data out and have a finished product that does not require X to run.
Even if one were to require open sourcing derived datasets it is usually easy to segregate the dataset with a tainted (open source) license as you build up your data so the new datasets are not formally "derived" and thus would not need open sourcing.
I would love a better way forward on this, or at least a cleaner explanation of options.
Re: Open Source Datasets
#16I have been thinking about what this solves in respect to other datasets. Nearly all shape recognition datasets have a restriction that you can't use unless you are an academic. I feel like that Open Sourcing data sets will allow us to be more democratic with data and the things that are generated by them. Creative Commons seems like a good license for this though. Once you have the data is half the battle . The rest…
> Creative Commons seems like a good license for this though I don't think that CC licenses are a good fit for data collections in general. Let's see what other open data projects do: Some open data projects were not satisfied with CC licenses, which is why ODbL was created: https://opendatacommons.org/licenses/odbl/ Also, many open data projects choose to put their stuff into the public domain (or license it as "CC0…
I agree that CC licenses aren't a good fit for data collections. I have been meaning to create a new license but I don't have the expertise or credibility to reasonably market it. ODbL seems really close to what I was envisioning. Thanks!
Re: Open Source Datasets
#17One question that is not clear to me is what should the dataset license to allow / restrict, in the perfect world. For me (just a personal opinion) it would allow free (as in liberty) use, but somehow encourage those who use it to share the benefits (data, software or algorithms) under the same license. Unfortunately, Open Source does not help here -- I do not see how OS can be used with data sets. The main OS levera…
OS helps tremendously in reproducibility. Without the underlying data, there is no way to audit an analysis. Moreover, an algorithm is only ever "done" in the same way that software is ever "done". New techniques might come along that could enhance the model, or the business requirements might change that necessitate re-tuning the algorithm.
The benefits of OS data are the same as the benefits of OS software. The distinction between "Free" and "Open" is the same as well.
Edit 1: OS data sets are nothing new. The UCI Machine Learning Repository[1] has been around for years. There is also an entire Open Data Stack Exchange site [2], and an Open Data Subreddit [3].
Edit 2: OS data sets are essential for developing new algorithms because they can be used as benchmarks. Nobody should trust a model that's been developed on a proprietary data set for use on anything other than that one data set.
[1]: https://archive.ics.uci.edu/ml/
Re: Open Source Datasets
#18I have been thinking about what this solves in respect to other datasets. Nearly all shape recognition datasets have a restriction that you can't use unless you are an academic. I feel like that Open Sourcing data sets will allow us to be more democratic with data and the things that are generated by them. Creative Commons seems like a good license for this though. Once you have the data is half the battle . The rest…
Nearly all shape recognition datasets have a restriction that you can't use unless you are an academic. I feel like that Open Sourcing data sets will allow us to be more democratic with data
Just because something is open source doesn't mean it can't have an academia-only restriction. Data sets should cost money in for-profit uses, open or not.
Re: Open Source Datasets
#19I have been thinking about what this solves in respect to other datasets. Nearly all shape recognition datasets have a restriction that you can't use unless you are an academic. I feel like that Open Sourcing data sets will allow us to be more democratic with data and the things that are generated by them. Creative Commons seems like a good license for this though. Once you have the data is half the battle . The rest…
Nearly all shape recognition datasets have a restriction that you can't use unless you are an academic. I feel like that Open Sourcing data sets will allow us to be more democratic with data Just because something is open source doesn't mean it can't have an academia-only restriction. Data sets should cost money in for-profit uses, open or not.
Academia doesn't cover all non-profit uses though. What about hobbyists?
Re: Open Source Datasets
#20The lab I work in publishes, and has continues to extend, the studyforrest dataset for quite a few years now.
Most of the consumers so far have been neuroscience researchers and statisticians, but we do hope (and think) that there's value for a wide variety of interests.
There's a bunch of different data, but the highlights are fMRI scans of people watching and/or listening to the movie Forrest Gump, eye tracking, and detailed annotations of the movie. We are also about to begin acquiring simultaneous EEG and fMRI.
http://studyforrest.org/data.html
Accessing the data is easy, and, as great admirers of Joey Hess, we also have it available in a git annex repo. :-)
http://studyforrest.org/access.html
---Alex
[EDIT] Given that this thread is about open source datasets, it's probably worth mentioning that the license is PDDL.