Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Millions of accounts compromised because there is no specialised user database

fourlightyears.blogspot.com

Post reply on HN
Print view View original

51–60 of 61 posts

Re: Millions of accounts compromised because there is no specialised user database

#51

Post by hyc_symas »

beagle3 wrote: post #48
bagels wrote: post #35

Earlier quoted context omitted.

Can existing LDAP implementations scale a single directory to hundreds of millions of users? I can find no clear answer after a few minutes of searching.

You can probably list the organizations (outside of China, perhaps) that have over a hundred million of users in a twitter message - (Facebook;Twitter;Google;Walmart;WhatsApp;Instagram;Hotmail;Snapchat;Yahoo;IRS[0];Blizzard;AOL;AT&T;Verizon) is probably more than 30% of the list already. And AFAIK not a single one of them uses LDAP - but assume one of them does. So, practically speaking - does it matter? [0] And equi…

You're quite wrong here. Google, AT&T and Verizon on your list use OpenLDAP. OpenLDAP scales to billions of users in a single instance, and of course it distributes across multiple nodes for even greater scaling as needed. Most of the telcos around the world use it now; it scales far beyond every other data storage technology out there.

http://symas.com/docs/2014-LOADays-jkozyra.pdf

80% of the US population that owns cellphones are OpenLDAP users, whether they realize it or not. 90% of the US population that subscribes to cable TV (Comcast, Time-Warner, etc.) as well. Nothing else actually works for hundreds of millions of users. These companies know, they've tried a lot of "solutions" already.

Re: Millions of accounts compromised because there is no specialised user database

#52

Post by hyc_symas »

viraptor wrote: post #32
Spooky23 wrote: post #7

There is such a beast, it's called LDAP. There are any number of directory and authorization services, and they have probably been around for decades. There are dozens of directory server options, and probably a few dozen secure authorization solutions, from Kerberos and SAML to proprietary solutions like SiteMinder. Together, these solutions give you EIAM capability that does what you described.

While technically you're right (as in, LDAP is great for handling many user entries and auth[nz] correctly), in practice it's not going to happen. Nobody knows LDAP (as in, take a random sample of non-corporate developers and most likely none of them will know LDAP). It's a pain to integrate and operate in a small environment. That's why there's a call for a simple solution. It's why everyone uses JSON and not ASN.1.…

Buzz, wrong - no such thing as LDAP injection attacks, sorry.

Re: Millions of accounts compromised because there is no specialised user database

#53

Post by hyc_symas »

andrewstuart wrote: post #9
Spooky23 wrote: post #7

There is such a beast, it's called LDAP. There are any number of directory and authorization services, and they have probably been around for decades. There are dozens of directory server options, and probably a few dozen secure authorization solutions, from Kerberos and SAML to proprietary solutions like SiteMinder. Together, these solutions give you EIAM capability that does what you described.

My uninformed perception of OpenLDAP is that it is a large, generalised enterprise class directory service that does alot of stuff. The post suggests a minimal, lightweight user database that does nothing at all else - is that the same thing as OpenLDAP?

OpenLDAP comes with a lot of optional modules to implement enterprise directory features. You don't need to use them all. You can build a minimal OpenLDAP system that's dedicated to user authentication that runs in under 4MB of RAM, with full TLS/SASL/GSSAPI security. You can't even do that with SQLite, the SQLite binary alone is 2MB and that doesn't give you any network interface or security features.

Re: Millions of accounts compromised because there is no specialised user database

#54

Post by derefr »

BinaryIdiot wrote: post #16

This entry really boils down to separating user data from other data. But some of the items I don't think work in real usage (or at least I would like some clarification if possible). > It should be accessible only via its specialised API which is designed to constrain the ways that it is accessed. Constrained in what ways? Especially in management / support roles you may need to do bulk actions against users (full l…

> Especially in management / support roles you may need to do bulk actions against users (full listing, bulk changes, etc) so I'm not sure what constraints would be compared to a normal database.

I was imagining a social-network use case while reading the post, so when a similar objection came to mind, the answer seemed obvious: make "users" and "accounts" separate things. The business-logic keeps around its own "users" table. The business logic can, and must, index that.

Each user then HAS A account; there's a 1:1 correspondence. The accounts are in a separate service (a AAA microservice, effectively) that exposes this minimal API where you can only talk about individual accounts to it.

How does a backoffice user-management dashboard do its job? Like the name suggests, by reading off the users table—and then doing O(N) queries to the accounts service to effect changes. These sort of one-off administrative operations are allowed to be costly like that.

Re: Millions of accounts compromised because there is no specialised user database

#55

Post by derefr »

danellis wrote: post #14
andrewstuart wrote: post #12

Earlier quoted context omitted.

Misguided? The microservice you suggest is precisely what the post suggests. The post suggests that the code of the microservice ensures that the correct hashing is used and that the data is stored correctly. It's puzzling that you say this is "completely misguided" and describe exactly the same thing.

OP called for "a specialized database". I was talking about using the same kind of database.

I don't know if you're really thought about it, but a microservice API is a database.

A Postgres database, for example, with correctly-limited user permissions set, and a good set of views and triggers and stored procedures, is exactly equivalent in power and flexibility to putting some blobs of Sinatra API-routing and model code in front of Postgres-used-as-a-dumb-object-store.

RDBMSes were always meant to do, on their own, exactly what microservices are doing now. That's why they're so much more complicated than just being dumb object stores: you can encode within a RDBMS schema the constraints of the model, and the valid transformations you can make to any given piece of data, to create an API exactly as narrow and high-level as a given user needs.

I think it was just some quirk of their use in mainframe contexts which lead to their auth protocols never being hardened to work on the public Internet, which meant people started to believe there was something inherently insecure about letting users talk to an RDBMS.

There's not; a database, a real database, is a persistent micoservice server.

And now we're all the way around to reinventing them on top of themselves, but this time they're three separate process-spaces, random custom languages are required, and they don't understand questions phrased as relational algebra.

Re: Millions of accounts compromised because there is no specialised user database

#56

Post by beagle3 »

hyc_symas wrote: post #51
beagle3 wrote: post #48

Earlier quoted context omitted.

You can probably list the organizations (outside of China, perhaps) that have over a hundred million of users in a twitter message - (Facebook;Twitter;Google;Walmart;WhatsApp;Instagram;Hotmail;Snapchat;Yahoo;IRS[0];Blizzard;AOL;AT&T;Verizon) is probably more than 30% of the list already. And AFAIK not a single one of them uses LDAP - but assume one of them does. So, practically speaking - does it matter? [0] And equi…

You're quite wrong here. Google, AT&T and Verizon on your list use OpenLDAP. OpenLDAP scales to billions of users in a single instance, and of course it distributes across multiple nodes for even greater scaling as needed. Most of the telcos around the world use it now; it scales far beyond every other data storage technology out there. http://symas.com/docs/2014-LOADays-jkozyra.pdf 80% of the US population that owns…

Then I'm glad to stand corrected - I suppose I'm horribly out of date from the days they tried other "solutions". I'm less surprised about AT&T and Verizon, more surprised about Google.

My own personal experience with LDAP in general (and some with OpenLDAP in particular) dates back to 2000 or so -- and at some point I just gave up on the complex and unreliable beast that it was (for whatever reason, BDB would occasionally lock up or get corrupted) - but I know a lot has changed since, LMDB is amazing -- I guess it is time for another look.

Re: Millions of accounts compromised because there is no specialised user database

#57

Post by viraptor »

hyc_symas wrote: post #52
viraptor wrote: post #32

Earlier quoted context omitted.

While technically you're right (as in, LDAP is great for handling many user entries and auth[nz] correctly), in practice it's not going to happen. Nobody knows LDAP (as in, take a random sample of non-corporate developers and most likely none of them will know LDAP). It's a pain to integrate and operate in a small environment. That's why there's a call for a simple solution. It's why everyone uses JSON and not ASN.1.…

Buzz, wrong - no such thing as LDAP injection attacks, sorry.

Come on, it's just one google search away. https://www.owasp.org/index.php/LDAP_injection (but confirms the "nobody knows LDAP" nicely)

Unless you escape your "(cn=$username)" properly, the users can do a lot of fun things.

Re: Millions of accounts compromised because there is no specialised user database

#58

Post by hyc_symas »

viraptor wrote: post #57
hyc_symas wrote: post #52

Earlier quoted context omitted.

Buzz, wrong - no such thing as LDAP injection attacks, sorry.

Come on, it's just one google search away. https://www.owasp.org/index.php/LDAP_injection (but confirms the "nobody knows LDAP" nicely) Unless you escape your "(cn=$username)" properly, the users can do a lot of fun things.

Nonsense. That page is utter tripe. The 2nd example " ( cn = jonys ) ( | (password = * ) ) " isn't even a valid filter, and a proper LDAP server will discard it. An OpenLDAP libldap-based app won't even be able to generate the request because the library will reject it.

More importantly, an LDAP search filter only goes with an LDAP search request, which is read-only. You cannot use silly parser games like this to destroy a database, like you can in SQL with Little Bobby Tables. https://xkcd.com/327/

Playing games with LDAP search filters can't return you more information than the app was originally authorized to see. It most often won't even return you more than a single record, since most apps that make LDAP queries and expect only a single entry to be returned won't even look at any subsequent entries if they happen to be in the reply.

Canonical LDAP application code to search for a user record, for authentication/authorization purposes, sets the LDAP search sizelimit to 1. Getting more than 1 matching result is an error.

Re: Millions of accounts compromised because there is no specialised user database

#59

Post by hyc_symas »

beagle3 wrote: post #56
hyc_symas wrote: post #51

Earlier quoted context omitted.

You're quite wrong here. Google, AT&T and Verizon on your list use OpenLDAP. OpenLDAP scales to billions of users in a single instance, and of course it distributes across multiple nodes for even greater scaling as needed. Most of the telcos around the world use it now; it scales far beyond every other data storage technology out there. http://symas.com/docs/2014-LOADays-jkozyra.pdf 80% of the US population that owns…

Then I'm glad to stand corrected - I suppose I'm horribly out of date from the days they tried other "solutions". I'm less surprised about AT&T and Verizon, more surprised about Google. My own personal experience with LDAP in general (and some with OpenLDAP in particular) dates back to 2000 or so -- and at some point I just gave up on the complex and unreliable beast that it was (for whatever reason, BDB would occasi…

OpenLDAP didn't start to become seriously usable until about 2003 (that's when HP dropped SunDS and adopted OpenLDAP for all of their global infrastructure). A lot of work went into it in the 2001-2002 timeframe to get it there. Yeah, a lot has changed, you've missed a couple generations of advancements. BDB's complexity was definitely a pain, but that's all in the past.

Re: Millions of accounts compromised because there is no specialised user database

#60

Post by viraptor »

hyc_symas wrote: post #58
viraptor wrote: post #57

Earlier quoted context omitted.

Come on, it's just one google search away. https://www.owasp.org/index.php/LDAP_injection (but confirms the "nobody knows LDAP" nicely) Unless you escape your "(cn=$username)" properly, the users can do a lot of fun things.

Nonsense. That page is utter tripe. The 2nd example " ( cn = jonys ) ( | (password = * ) ) " isn't even a valid filter, and a proper LDAP server will discard it. An OpenLDAP libldap-based app won't even be able to generate the request because the library will reject it. More importantly, an LDAP search filter only goes with an LDAP search request, which is read-only. You cannot use silly parser games like this to des…

Uncommon - yes. Can do less than SQL injections - yes. Read-only - yes.

But it does exist. Saying that LDAP auth[nz] code should set sizelimit to 1 is like saying SQL auth[nz] code should set LIMIT 1. Correct and defensive code does that. But there's still code out there which doesn't and is vulnerable.

Then again, I don't think you understood the second example. It's not for auth, but rather user search - which is the probably the second most common corporate ldap usage.

As for LDAP injection being nonsense, how about searching some CVEs:

- LDAP injection and resulting STORED Cross Site Scripting in Events Planner - SmarterMail 7.x (7.2.3925) - https://www.exploit-db.com/exploits/15189/

- Graylog2 before 0.92 allows remote attackers to bypass LDAP authentication via crafted wildcards. - CVE-2014-9217

- PowerDNS before 2.9.18, when running with an LDAP backend, does not properly escape LDAP queries, which allows remote attackers to cause a denial of service (failure to answer ldap questions) and possibly conduct an LDAP injection attack. - CVE-2005-2301

And many many more...

Post reply on HN