Ask HN: Securely store sensitive data in the DB? • Hacker News Board

Re: Ask HN: Securely store sensitive data in the DB?

#41

Post by stan_rogers » Sun, Jul 28, 2013, 4:47 PM UTC

rythie wrote: post #32

Here's an idea: Use the user's password to decrypt a key, that then decrypts the data - which I know you can't do because of password resets... So to deal with password resets, create another password which decrypts the same key. Store that other password in a physical safe, possibly in a bank safety deposit box. This will slow down password resets to a manual process of course. For additional security you can store…

Trivia note: this is, in a nutshell, how the Lotus/IBM Notes ID works. The password is used in a KDF to generate a key, which in turn decrypts the user's private key (and certain other credentials, along with symmetric secret keys for shared encrypted doccuments). Success/failure is determined solely by the successful decryption of known bytes in the encrypted package. Other info (the user's public key, identity and certifier, all signed) are maintained in the clear and can be easily and safely exported and may be "trusted" for authentication with remote machines. There is a "password recovery" system as well (it doesn't actually recover the password, but allows a reset), requiring cooperation of two or more admins¹ (in a Shamir-type arrangement) so that previously-encrypted user data will not be lost.

¹ There is the option to use a single admin, but there are great big warning signs and scary red boxes all over that section of the doco. It's something you'd only use in a solo shop (as a Notes ISV or a Domino web dev).

Re: Ask HN: Securely store sensitive data in the DB?

#42

Post by jahewson » Sun, Jul 28, 2013, 5:28 PM UTC

The best practice for storing sensitive data: don't. You have to design your system presuming that someone has hacked your server - if the encryption key is stored there, they can find it.

- As you're aware, password hashes are a way to avoid storing sensitive data (though they should still be treated as sensitive). You're using a strong hash function and a salt, right?

- Re-use the principle of password hashes for API keys to simply avoid having to store hyper-sensitive data: generate a long (say 512-bit) secure random number (using OpenSSL) as the user's secret API key. Then hash the key as if it were a password and store only the hash. Now if someone steals your API key database they can't use it to authenticate as your users.

Note: for API keys a strong hash such as bcrypt will probably be too slow and resource-intensive. However, because API keys are (long) random data, unlike passwords, you can use a faster hash function like SHA-1.

- As for credit card data: don't. You probably can't afford the PCI audits and dedicated hardware and the same principle applies: just don't store sensitive data. Instead, many payment gateways offer 'tokens' for recurring payments in which you pass the payment information to their API without storing it (or use their hosted page in iframe, if acceptable to you) and they return a token which can be used to charge against that card in the future. Not all payment gateways offer this, and some charge (too much) so take a look at https://spreedly.com/ which offers a middle-man gateway service which adds tokens and other API feature to pretty much all of the payment gateways.

As you can see, in both cases it's possible to simply avoid storing the most sensitive data.

Re: Ask HN: Securely store sensitive data in the DB?

#43

Post by tomjen3 » Sun, Jul 28, 2013, 6:31 PM UTC

patio11 wrote: post #9

RyanZAG wrote: post #7

I'd say the first thing to understand here is that absolute safety is impossible in this case. If the hosting server is compromised, password loggers can be installed and even the login page itself can be altered to remove any form of security. With access to emails, an attacker could send an official email asking everyone to reset their passwords, etc. So your question is actually: How can I make my system divulge t…

This gives the attacker an additional server he would need to hack. If they root your web tier, and your web tier knows how to ask your internal service layer for sensitive data, then the attacker knows how to ask your internal service layer for sensitive data. I really hate repeating "If you lose any one box in your deployment then you can assume you will lose all data, regardless of whether you encrypt things or no…

Wouldn't that be a simple issue of limiting what the web layer can ask for? I mean sure it would allow the attacker to charge a creditcard - but hopefully only to an approved account, and he wouldn't ever see the actual details. Your web layer never needs them, so why should it have the right to ask for them?

Re: Ask HN: Securely store sensitive data in the DB?

#44

Post by nandemo » Mon, Jul 29, 2013, 12:07 AM UTC

raverbashing wrote: post #2

- Keep passwords in memory (so as if you start the service it prompts for the password) - Asymmetrical crypto. So for example, you encrypt your CC data upon sign-up but then to run the charges you need the private key (and this is somewhere else) - Enable SSL communication with your DB. Postgres has this, because being defeated by network sniffing is bad.

In the first case, what if we're running a service in a cluster that starts instances automatically?

Re: Ask HN: Securely store sensitive data in the DB?

#45

Post by a_smith » Mon, Jul 29, 2013, 10:18 AM UTC

I've recently been looking into the same issue. I need a way to encrypt data before inserting it into a database in such a way that the person inserting the record can read it, their supervisor can read, but their colleague can't. It needs to survive a password reset and I don't want to store any keys on the server unencrypted.

This lead me to attribute based encryption and the libbswabe library. The idea is you generate a master keypair and from these you generate private keys for each of your users. Your user's private keys can only decrypt data that was encrypted with attributes that were also applied to their key.

For example, let's say we have 2 users Alice and Bob. Alice is a supervisor for the IT department. Her key was generated with the attributes "alice" (her username) and "itdepartment". Bob is a normal employee in the IT department, the only attribute applied to his private key was his username "bob"

Now lets say we use the master public key to encrypt each of the fields in the user table (Firstname, Lastname, Email, etc). If each field for a user record is encrypted with the attributes: [current_username] and "itdepartment", then Bob can decrypt his fields because they are tagged with "bob" and "bob" is an attribute in his key and Alice can decrypt her record through the same logic AND every record whose fields were encrypted with the attribute "itdeparment".

If users private keys are encrypted with their password and stored in the database, then the only way you can get Bob's key is to break his password. An attacker now has access to the data that Bob's key can decrypt, but importantly, not everything. If Bob forgets his password (and therefore can't access his private key) then a new one can be generated and all it needs to do is have the "bob" attribute in order for him to have access to all his old data.

Now this is by no means a complete description of a solution, you have to securely store the master private key (you only need this to generate private keys for your users though, not for every put/get request), there's issues around key revocation and lots of gaps in my description, but these issues are present for any crypto system. Attribute based encryption though seems to me like it overcomes a lot of the issues that plague other solutions, the biggest single one being that other solutions require the master private key to either be on disk, or in memory at all times, this solution doesn't need that.

Re: Ask HN: Securely store sensitive data in the DB?

#46

Post by ra » Mon, Jul 29, 2013, 9:47 PM UTC

raverbashing wrote: post #33

ra wrote: post #12

Earlier quoted context omitted.

Memory is not secure. It's quite a common attack to grab keys / passwords from the memory of an executing program.

No, it is one of the safest places. If your attacker has access to arbitrary memory of a process, you're using an insecure OS/version. Or they dumped your process memory using a vulnerability (in your system) Yes, there are some possible attacks (page file, cache, etc) Attacking memory after a reboot requires physical access (unless you hibernated without an encrypted file, in this case...) It certainly beats the sec…

Just because it's more secure than network / file doesn't mean it's secure. That's why we have smartcards / HSMs.

Re: Ask HN: Securely store sensitive data in the DB?

#47

Post by anywherenotes » Tue, Jul 30, 2013, 1:35 PM UTC

I was just at Oracle 12c presentation, and I'm not pushing for Oracle (and not associated with them), but just going to mention something they have, that maybe your database provider also has.

Oracle (I believe starting from 12c) has masking for data. They were saying how you can mask everything except last 4 digits of credit cards. So if someone gains credentials of a regular employee, they will be able to query data, but sensitive data will be masked by database itself.

Re: Ask HN: Securely store sensitive data in the DB?

#48

Post by raverbashing » Tue, Jul 30, 2013, 2:16 PM UTC

ra wrote: post #46

raverbashing wrote: post #33

Earlier quoted context omitted.

No, it is one of the safest places. If your attacker has access to arbitrary memory of a process, you're using an insecure OS/version. Or they dumped your process memory using a vulnerability (in your system) Yes, there are some possible attacks (page file, cache, etc) Attacking memory after a reboot requires physical access (unless you hibernated without an encrypted file, in this case...) It certainly beats the sec…

Just because it's more secure than network / file doesn't mean it's secure. That's why we have smartcards / HSMs.

Smartcards are vulnerable as well and there has been successful attacks towards some smartcards (google it)

And not every system has an HSM available

Re: Ask HN: Securely store sensitive data in the DB?

#49

Post by ra » Wed, Jul 31, 2013, 11:00 AM UTC

raverbashing wrote: post #48

ra wrote: post #46

Earlier quoted context omitted.

Just because it's more secure than network / file doesn't mean it's secure. That's why we have smartcards / HSMs.

Smartcards are vulnerable as well and there has been successful attacks towards some smartcards (google it) And not every system has an HSM available

Sure but smartcards and HSMs are slow and in a properly managed environment are much safer than memory.

Whether or not the OP has an HSM is moot. The OP said, "I want to design and implement a solution as secure as it can be" ... and that means (among many other things) keys on HSM.

Re: Ask HN: Securely store sensitive data in the DB?

#50

Post by ProblemFactory » Wed, Jul 31, 2013, 6:58 PM UTC

patio11 wrote: post #9

RyanZAG wrote: post #7

I'd say the first thing to understand here is that absolute safety is impossible in this case. If the hosting server is compromised, password loggers can be installed and even the login page itself can be altered to remove any form of security. With access to emails, an attacker could send an official email asking everyone to reset their passwords, etc. So your question is actually: How can I make my system divulge t…

This gives the attacker an additional server he would need to hack. If they root your web tier, and your web tier knows how to ask your internal service layer for sensitive data, then the attacker knows how to ask your internal service layer for sensitive data. I really hate repeating "If you lose any one box in your deployment then you can assume you will lose all data, regardless of whether you encrypt things or no…

While this is true in theory, compartmentalizing services and data can add security in practice. Attackers may not know the details your internal systems, and any unusual behaviour helps with detecting intrusions.

* For example, if credit card details are stored on a separate service, then the web tier can be given only a "charge the card" API, instead of a full read access.

* Even if the decryption key is held in memory, encrypting the data on disk helps against accidental leaking of backups, physical theft of the servers. Or a hacker with limited skills who copies a database dump, is discovered from the unusual network load, and does not have time to fully investigate the system and extract the key from memory.

* If sensitive data is held on a separate service, but the web tier has read access to it - then the other service can impose rate limiting and unusual activity detection to block attempts to dump everything quickly.

* If the data is encrypted with the user's password, which is not stored in plaintext at all - then the attacker can at best only access accounts that log in before detection.

All of these still give an attacker full access, assuming they have infinite amount of time and skills. But for many practical scenarios, they can reduce the amount of compromised data.