Ethical issues in research using datasets of illicit origin

Is the demand for it that is creating the rise of illicit data collections.

It may be a wonderful paper and discussion but the author seems ridiculously positive about IRBs.

http://slatestarcodex.com/2017/08/29/my-irb-nightmare/

HN discussion of the above

https://news.ycombinator.com/item?id=15127271

There is a very fine line between authorized data, technically public but implicitly unauthorized data, and illegally obtained, unauthorized data. Here’s an example of each in the financial sector, from my personal experience:

1. Financial account aggregators and “budget apps” like Min monetize their business, in part, by selling huge amounts of data to the financial sector. Sometimes companies like Second Measure take raw data from companies like Yodlee and clean it, then resell it. Nowadays there is an entire industry of alterative market research that has had all sorts of participants, from Foursquare (locations) to Spark (email enhancement). This is technically authorized, because it’s in the TOS. The users effectively contribute their own data.

2. I developed an extremely accurate, reasonably generalizable method of forecasting vehicle production at several companies that relies on implementing a VIN searching algorithm in conjunction with legally required NHTSA recall lookup portals hosted by each manufacturer. This data is what you’d call unauthorized, because no entity explicitly endorses your use of it. For example, several colleagues and I knew well ahead of time that Tesla would miss on production of the Model 3s because they were utterly unrepresented in our data. But this data is public, so it’s fine to use from a legal and compliance standpoint. It was lucrative data specifically because it had a high signal for revenue, yet was hitherto unused and unidentified.

3. I once found, in the course of looking for legally usable data, an actual security vulnerability disclosing all users of a publicly traded QSR’s online delivery service, along with their phone numbers, email addresses and last four digits of credit cards. This is both unauthorized and illegal, because the data is contaminated with personally identifiable information and it clearly requires a vulnerability (not just scraping) to acquire.

I’ve seen overzealous data vendors accidentally slip from #2 into #3, which is really bad for all concerned. It’s not a great look for the vendor, who will likely be fired, and it represents a breach for the company who owns the data and its users. Any firm that has purchased the data will likely be contamined and be forced into a trading lockdown of that security for a period of time by compliance.

My real concern is that illicit data like this is used in machine learning research. Machine learning is already pretty frustrating - it’s common for me to find research from a conference that I’m simply unable to replicate because the training or experiment data is not available (this is annoyingly the case with A/B experiment optimization research put out by giant companies in particular). I worry that this trend of accepting machine learning research without any requirement for total data transparency will incentivize researchers to conduct their experiments using illicit data that doesn’t need to be sourced.

I love that story. But I have two questions.

1) What would be the possible motivation for such hostility from the first reviewer?

2) Why did you create a temporary throwaway account but then promote a paper with your real name and information?

1) The reviewer may have had similar work and wanted to hold up the publishing of OP’s paper while finishing their own work. Or there’s some personal animus. Or OP’s paper may have threatened to surplant the reviewers work. There’s a whole plethora of reason for a particularly hostile review. It’s a big enough problem that my partner who’s going through grad school now had the option of requesting specific people NOT be on the reviews for her (first publishing in gradschool as a first author!) paper because of scooping, animus towards her PI, etc. that may have resulted in an overly hostile review.