Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A raw dump of companies from all over the world by LinkedIn handle

blog.bigpicture.io

Post reply on HN
Print view View original

61–70 of 108 posts

Re: A raw dump of companies from all over the world by LinkedIn handle

#61

Post by mfrye0 »

paxys wrote: post #47

You use "open source" multiple times in the post, HN title, HN comments, but: 1. The source code for the project isn't shared anywhere. 2. The data isn't shared under any standard open source license. 3. The terms of your site explicitly prohibit commercial use of this data. So what exactly makes this "open source in the broadest sense"?

Fair points. I agree the wording could be better.

No, the source code is not available. This dataset is a subset of the raw data our system collects. Our final product made available via the API does a variety of processing steps on the raw data (dedupes, joins, ML predictions, etc). The final, processed data is the piece that is proprietary / subject to the terms.

We will update the site terms to reference this dataset, as we aim to continue releasing an updated version each quarter. I'll have to double check with the lawyers, but it will most likely be MIT licensed.

Re: A raw dump of companies from all over the world by LinkedIn handle

#63

Post by noizejoy »

mcenedella wrote: post #51
babblingfish wrote: post #27

It's funny how OP does not address where this data comes from even though it's obviously from LinkedIn. I see many people in the comments asking questions so I will add my two cents as someone who is currently employed by LinkedIn and has an interest in web scraping. This dataset was taken from scraping the company pages from LinkedIn. A company has to pay to have this page, so this certainly does not include all com…

LinkedIn lost the HiQ case. Your GC claiming it’s a win doesn’t change that you lost, repeatedly, in multiple courts of law: https://en.m.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn

> LinkedIn lost the HiQ case. Your GC claiming it’s a win doesn’t change that you lost, repeatedly, in multiple courts of law

That would appear to be false, since after appeals, there was a settlement wich would seem to make it pretty clear that hiQ lost:

https://www.natlawreview.com/article/hiq-and-linkedin-reach-...

Re: A raw dump of companies from all over the world by LinkedIn handle

#64

Post by time_to_smile »

paxys wrote: post #47

You use "open source" multiple times in the post, HN title, HN comments, but: 1. The source code for the project isn't shared anywhere. 2. The data isn't shared under any standard open source license. 3. The terms of your site explicitly prohibit commercial use of this data. So what exactly makes this "open source in the broadest sense"?

It's open source in the sense of OSINT [0]. Clearly confusing on a site like Hacker News, but this has been standard usage of the term for that community for a long time now.

0. https://en.wikipedia.org/wiki/Open-source_intelligence

Re: A raw dump of companies from all over the world by LinkedIn handle

#67

Post by dang »

The submitted title was "World's largest open source company dataset", but (1) "world's largest" is linkbait and the article walks it back, (2) "open source" could be worded better per https://news.ycombinator.com/item?id=35979581, and (3) the only thing left in the title after taking those out would be "company dataset", which is too generic to be a good title.

I've therefore replaced the title above with what appears to be an accurate description from https://news.ycombinator.com/item?id=35978156.

Re: A raw dump of companies from all over the world by LinkedIn handle

#68

Post by slabity »

mfrye0 wrote: post #61
paxys wrote: post #47

You use "open source" multiple times in the post, HN title, HN comments, but: 1. The source code for the project isn't shared anywhere. 2. The data isn't shared under any standard open source license. 3. The terms of your site explicitly prohibit commercial use of this data. So what exactly makes this "open source in the broadest sense"?

Fair points. I agree the wording could be better. No, the source code is not available. This dataset is a subset of the raw data our system collects. Our final product made available via the API does a variety of processing steps on the raw data (dedupes, joins, ML predictions, etc). The final, processed data is the piece that is proprietary / subject to the terms. We will update the site terms to reference this data…

What exactly does "open-source" mean to you? Because it sounds like there's absolutely nothing open about this other than a small scraping of LinkedIn data (which you should probably ask your lawyers if you're even allowed to license out).

The wording isn't just misleading, it's a complete lie.

EDIT: Nevermind, the title has been updated to accurately reflect this being a small datadump.

Re: A raw dump of companies from all over the world by LinkedIn handle

#69

Post by companydataguy »

This is Duedil + Company Check + Open Corporates.

Duedil and CC were (mostly) powered by Creditsafe data which is much better in Europe at least than D&B

Open Corporates sold their data to Creditsafe for low 5 figures.

Interesting point re DnB in Eu it’s mostly a license if the brand name and owns little of the data or the business.

Re: A raw dump of companies from all over the world by LinkedIn handle

#70

Post by NoboruWataya »

mikecoles wrote: post #10
data_maan wrote: post #4

I looked at the attributes they say the dataset has. Not too many (e.g. number of people, location). The really interesting ones, like who is doing business with whom, are missing.

I'd like to see parent company listed. How many brands fall under TTI or Unilever? For example, a hierarchy of tool brands: https://www.protoolreviews.com/power-tool-manufacturers-who-...

https://search.gleif.org/#/record/549300MKFYEKVRWML317

(Not brands, but shows you subsidiaries.)

Post reply on HN