Live data from Hacker News

Personal and social information of 1.2B people discovered in data leak

dataviper.io

151–160 of 440 posts

Re: Personal and social information of 1.2B people discovered in data leak

#151

Earlier quoted context omitted.

You forgot the part about exposing your finished database to unprotected elasticsearch http endpoint ;) In all seriousness does anyone know why you can even host an elasticsearch database as http and without credentials? Seems to be the default. What is the use case for this?

The use case is in a local datacenter, with a NAT-ed IP not exposed to the main web

A firewalled IP would be much more appropriate, and NAT is not a firewall or a security mechanism.

Re: Personal and social information of 1.2B people discovered in data leak

#152
post #131

I found a vulnerability in linkedIn a few years back that allowed anyone to access a private profile (because client side validation was enough for them I guess..?) They didn't take my report seriously (still not completely patched) and I feel like that told me all I needed to know about their security practices.

I had a similar experience. In 2014 I reported an issue where you could take over someone's account by adding an email you control to it and having them complete the flow by sending them a link (which, unless they looked very carefully, looked exactly like the regular log-in flow at the time - especially if they used a public email service and you registered a similar-looking account). I tried it on a friend and it w…

LI is terrible if you actually try to use it, but it's harmless enough if you just use it as a profile hosting service, where people are likely to look. I just auto-archive their emails and only visit the site a couple of times per year.

Re: Personal and social information of 1.2B people discovered in data leak

#153

Earlier quoted context omitted.

I wrote a chrome headless framework that types using semi-realistic key presses (timing, mistakes, corrections) and does semi-realistic scrolling / swiping and clicking / tapping. It's not very hard to get something that would be too hard for almost every website beside Google and Facebook to bother with. If it's a 1 on a 0-9 scale in difficulty, most websites just don't have the resources to detect it It took me lik…

I think there's also a lot of bot-detection-as-a-service around here that can be used by sites smaller than Google and Facebook, like WhiteOps or IAS anti-fraud.

These are highly questionable under GDPR, many of them rely on tracking users wherever they go (e.g. Recaptcha is known for this).

Re: Personal and social information of 1.2B people discovered in data leak

#154

People data labs's data is pretty accurate. Here is mine: https://api.peopledatalabs.com/v4/person?api_key=9c6a1382204... You can try it for yourself by changing the email. All of the information is public, so I don't mind. They are basically doing data integration.

Your api key is now permanently in public. After few days, people will still be able to use this for their own usage.

Re: Personal and social information of 1.2B people discovered in data leak

#155
I was at an Elasticsearch meetup yesterday where we had a good laugh about several similar scandals in Germany recently involving completely unprotected Elasticsearch running on a public IP address without a firewall (e.g. https://www.golem.de/news/elasticsearch-datenleak-bei-conrad..., in German). This beats any of that.

Out of the box it does not even bind to a public internet address. Somebody configured this to 'fix' that and then went on to make sure the thing was reachable from the public internet on a non standard port that on most OSes would require you to disable the firewall or open a port. The ES manual section for network settings is pretty clear about this with a nice warning at the top: "Never expose an unprotected node to the public internet."

Giving read access is one thing. I bet this thing also happily processes curl -X DELETE "http::9200/*" (deletes all indices). Does it count as a data breach when somebody of the general public cleans up your mess like that?

In any case, Elasticsearch is a bit of a victim of its own success here and may need to act to protect users against their own stupidity since clearly masses of people who arguably should not be taking technical decisions now find it easy enough to fire up an Elasticsearch server and put some data in it (given the amount of companies that seem to be getting caught with their pants down).

It's indeed really easy to setup. But setting it up properly still requires RTFMing, dismissing the warning above, and having some clue about what ip addresses and ports are and why having a database with full read write access on a public ip & port is a spectacularly bad idea.

Re: Personal and social information of 1.2B people discovered in data leak

#156

People data labs's data is pretty accurate. Here is mine: https://api.peopledatalabs.com/v4/person?api_key=9c6a1382204... You can try it for yourself by changing the email. All of the information is public, so I don't mind. They are basically doing data integration.

Nothing for most of my accounts, except one which somehow was falsely attributed to someone else. Odd given I do have a LinkedIn profile; Their scraping must be far from perfect.

Re: Personal and social information of 1.2B people discovered in data leak

#157

Earlier quoted context omitted.

Let's be realistic here. Everyone knows it's not possible to cancel Comcast service.

"Ah yes, cancelling requires a call because of security. A feature for the user!"

To be fair, internets would have been equally outraged if there wasn't such requirement, because sure as hell somebody would have found an exploit and cancelled a bunch of account, just for funzies

Re: Personal and social information of 1.2B people discovered in data leak

#158
post #4

Yet another Elasticsearch server wide open. This is going to make the flurry of open mongodb servers look trivial.

I believe Elasticsearch doesn't allow restricting access by requiring login unless you pay for the enterprise version, which is just straight up stupid.

Re: Personal and social information of 1.2B people discovered in data leak

#159
post #109

People data labs's data is pretty accurate. Here is mine: https://api.peopledatalabs.com/v4/person?api_key=9c6a1382204... You can try it for yourself by changing the email. All of the information is public, so I don't mind. They are basically doing data integration.

This seems exceptionally unethical

Displaying public information publicly, or sharing your API key?

Re: Personal and social information of 1.2B people discovered in data leak

#160
post #17

Out of curiosity how do you guys think they managed to scrape LinkedIn on such a large scale? I've been wanting to do some social graph experimentation on it (small scale - say 1000 people near me) but concluded I probably couldn't scrape enough via raw scraping without freaking out their anti-scraping. (And API is a non-starter since that basically says everything is verboten).

So far, the answers have contained non-technical answers like "Distributed Scraping." Well, yes, obviously. A more useful answer is: I did this once, many years ago. Back then it was a matter of hooking up PhantomJS and making sure your user string was set correctly. Since PhantomJS was – I think – essentially the same as what headless chrome is today, the server can't determine that you're running a headless browser…

LinkedIn doesn't protection doesn't seem to be that sophisticated at the moment. Someone I know maintains ~weekly up-to-date profiles of a few million users via a headless scraper that uses ~10 different premium accounts and a very low number of different IPs.
Post reply on HN