Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Concept of a marketplace for machine learning datasets

datapie.in

Post reply on HN
Print view View original

31–40 of 46 posts

Re: Show HN: Concept of a marketplace for machine learning datasets

#31

Post by jamesblonde »

bussiere wrote: post #24
jamesblonde wrote: post #17

Earlier quoted context omitted.

Good points. We are soon going to release a p2p system for sharing datasets, backed by Hadoop clusters. You install the Hadoop stack (localhost or distributed), then you can free-text search for datasets that have been made 'public' on any hadoop cluster that participates in the 'ecosystem'. We expect it to be self-policing, but there will be a way to report illegal distribution of datasets. The solution is based on…

Will you make a ranking system for noting user and dataset ? I recommand using true skill for noting user and dataset. Keep up the good job.

It appears TrueSkill is patented - https://en.wikipedia.org/wiki/TrueSkill

Re: Show HN: Concept of a marketplace for machine learning datasets

#32

Post by minimaxir »

whyileft wrote: post #25
minimaxir wrote: post #7

There are a few services popping up with aim to provide data repositories for analysis/ML (Kaggle, data.world, /r/datasets) As someone who likes making analyses from random datasets, I have a few issues with these types of services: 1) There is often no indication of the distribution rights of the data, or whether the data was obtained ethically from the source (i.e. following the ToS). I made this mistake when I use…

Is there actual case law that prohibits the use of copyrighted material for corpora and other training data? Sure distribution can have issues, but do you have any references for simple possession as training and test data?

If you're gathering data for your own business, it would be difficult for anyone to know, sure.

But if the data/analysis is published, then the data source would need to be disclosed.

See more on the OKCupid case I mentioned above: http://www.vox.com/platform/amp/2016/5/12/11666116/70000-okc...

Re: Show HN: Concept of a marketplace for machine learning datasets

#33

Post by bussiere »

jamesblonde wrote: post #31
bussiere wrote: post #24

Earlier quoted context omitted.

Will you make a ranking system for noting user and dataset ? I recommand using true skill for noting user and dataset. Keep up the good job.

It appears TrueSkill is patented - https://en.wikipedia.org/wiki/TrueSkill

Or a ELO system

Re: Show HN: Concept of a marketplace for machine learning datasets

#34

Post by amelius »

What I like more is the concept of a job-agency for AIs. Basically, the job-agency is a broker between people who have data and need an algorithm, and people who have an algorithm but no data. The broker can then work as a matchmaker, but also provide protection against data/algorithm theft by managing hardware themselves.

As an example, see [1].

[1] http://www.aigency.co/

Re: Show HN: Concept of a marketplace for machine learning datasets

#35

Post by pgroves »

I swear this is what Infochimps used to be, but now I don't really see a reference to it on their website. Except for a 404 when I click on "Resources" -> "Data Marketplace"[1]. I'm guessing that means they moved away from that business. Looks like they now focus on tools, not data itself.

[1] http://www.infochimps.com/marketplace

Re: Show HN: Concept of a marketplace for machine learning datasets

#37

Post by justinclift »

jamesblonde wrote: post #31
bussiere wrote: post #24

Earlier quoted context omitted.

Will you make a ranking system for noting user and dataset ? I recommand using true skill for noting user and dataset. Keep up the good job.

It appears TrueSkill is patented - https://en.wikipedia.org/wiki/TrueSkill

Ugh, that's a fairly egregious patent. It's literally of a mathematical method. :(

Re: Show HN: Concept of a marketplace for machine learning datasets

#38

Post by Palomides »

pgroves wrote: post #35

I swear this is what Infochimps used to be, but now I don't really see a reference to it on their website. Except for a 404 when I click on "Resources" -> "Data Marketplace"[1]. I'm guessing that means they moved away from that business. Looks like they now focus on tools, not data itself. [1] http://www.infochimps.com/marketplace

I think they pivoted and then got bought out, but yeah, I recall the same.

Re: Show HN: Concept of a marketplace for machine learning datasets

#39

Post by joe_the_user »

akshaynathr wrote: post #2

Hi everyone, While working on some of my projects involving Machine learning algorithms and deep neural networks i have found that there is a lack of training data sets in many areas. Also many of them are scattered throughout web, some are extremely huge for an individual to process etc.So i thought of this idea of having a marketplace for data ,structured for machine learning communities. It can be a one stop place…

I don't think you can make this work because even a fairly small cost for data produces all sorts of problems.

You would have to distribute all these datasets with a license and such a thing would restrict any users much more than you'd realize - sharing one's results would become a trickier matter just for one. Deciding if the data is good would be another thing since being able to sell stuff produces an incentive for uploading garbage.

Just as much, huge datasets are effectively going to be the product of many people's data. If some entity decides it's going to profit from that, how do the profits get divided, especially if there's no protocols for that division.

For example, should someone be able to sell my chest X-rays if I haven't agreed to it? A lot of large companies with big data sets would face far more issues if they were to be selling those. And considerations go on and on and on.

The free-wheel, free-sharing quality of present day machine has been a driving force in its progress and it would be a shame for that to go away. As can be seen with the Internet, freely available data can be incredible force and hopefully data will remain freely available.

Re: Show HN: Concept of a marketplace for machine learning datasets

#40

Post by _v7gu »

justinclift wrote: post #37
jamesblonde wrote: post #31

Earlier quoted context omitted.

It appears TrueSkill is patented - https://en.wikipedia.org/wiki/TrueSkill

Ugh, that's a fairly egregious patent. It's literally of a mathematical method. :(

Just say yours does lower bound of Wilson score confidence interval for a Bernoulli parameter [0]. It has been used for sorting shopping items by ratings for years.

[0] http://www.evanmiller.org/how-not-to-sort-by-average-rating....

Post reply on HN