Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Personal and social information of 1.2B people discovered in data leak

dataviper.io

Post reply on HN
Print view View original

71–80 of 440 posts

Re: Personal and social information of 1.2B people discovered in data leak

#71

Post by btrettel »

Thriptic wrote: post #49
Scoundreller wrote: post #19

Earlier quoted context omitted.

I want someone to start an opt-out service, where I send them $20, and they send a book of names by registered mail for opt-outs every month. An online opt-out system is too easy for them. I want each one to get a phone-book sized list of opt-outs every month. And the same for data requests. Someone that curates the data collectors, and sends them requests every month. Do you know which country’s “do not call” list I…

> I want someone to start an opt-out service, where I send them $20, and they send a book of names by registered mail for opt-outs every month This exists but it's not cheap: https://www.abine.com/deleteme/

I'm not rich but $129/year isn't bad. I'd hesitate mostly because I assume such services are scams.

Re: Personal and social information of 1.2B people discovered in data leak

#72

Post by iamsb »

Havoc wrote: post #17

Out of curiosity how do you guys think they managed to scrape LinkedIn on such a large scale? I've been wanting to do some social graph experimentation on it (small scale - say 1000 people near me) but concluded I probably couldn't scrape enough via raw scraping without freaking out their anti-scraping. (And API is a non-starter since that basically says everything is verboten).

You use a proxy botnet and route your scraping requests through that. Use something like hola proxy or crawlera for example.

Re: Personal and social information of 1.2B people discovered in data leak

#73

Post by kevin_thibedeau »

plorg wrote: post #11
manigandham wrote: post #3

This is all scrapped public social media data. No credentials or govt information. It's very easy to download or buy this data legally.

It appears to also contain information possibly acquired from other companies. For example, the author notes that he had attributed to him a phone number he had assigned to him by AT&T that he never used or shared.

Guarantee the fine print in the AT&T contract authorized them to share information with a third party.

Re: Personal and social information of 1.2B people discovered in data leak

#76

Post by zer0tonin »

onlyrealcuzzo wrote: post #67
sillysaurusx wrote: post #58

Earlier quoted context omitted.

So far, the answers have contained non-technical answers like "Distributed Scraping." Well, yes, obviously. A more useful answer is: I did this once, many years ago. Back then it was a matter of hooking up PhantomJS and making sure your user string was set correctly. Since PhantomJS was – I think – essentially the same as what headless chrome is today, the server can't determine that you're running a headless browser…

I wrote a chrome headless framework that types using semi-realistic key presses (timing, mistakes, corrections) and does semi-realistic scrolling / swiping and clicking / tapping. It's not very hard to get something that would be too hard for almost every website beside Google and Facebook to bother with. If it's a 1 on a 0-9 scale in difficulty, most websites just don't have the resources to detect it It took me lik…

I think there's also a lot of bot-detection-as-a-service around here that can be used by sites smaller than Google and Facebook, like WhiteOps or IAS anti-fraud.

Re: Personal and social information of 1.2B people discovered in data leak

#77

Post by guenthert »

BoorishBears wrote: post #52
anilshanbhag wrote: post #31

People data labs's data is pretty accurate. Here is mine: https://api.peopledatalabs.com/v4/person?api_key=9c6a1382204... You can try it for yourself by changing the email. All of the information is public, so I don't mind. They are basically doing data integration.

Haha, when I was a kid and scared to use my real name for things, for some reason I used my email... which had my real name in it, to open a Github account with a fake name So the api knows me as the famous architect, Art Vandelay

That must have been a long time ago, Boorish Bears.

Re: Personal and social information of 1.2B people discovered in data leak

#79

Post by ars »

I don't know about other people, but I have zero personal info with LinkedIn and Facebook.

They only info they have about me is info I don't mind being public. If I want something to be private I don't tell it to them. It's as simple as that.

Google on the other hand, knows lots of private things.

Re: Personal and social information of 1.2B people discovered in data leak

#80

Post by SirYandi »

netsharc wrote: post #35
anilshanbhag wrote: post #31

People data labs's data is pretty accurate. Here is mine: https://api.peopledatalabs.com/v4/person?api_key=9c6a1382204... You can try it for yourself by changing the email. All of the information is public, so I don't mind. They are basically doing data integration.

Ah, looks like everyone's using that API key, I got 2 queries for my addresses and got a "rate limit exceeded" message. Strangely it only says I work in real estate (no I don't) when I looked up the email address I use for LinkedIn...

You, and others can use my api key, just signed up.

e75ac28b25480e60071b24d819d4692a0b315c037046b9ff6ec9dfb1e99a895c

Post reply on HN