Live data from Hacker News

Searchable field-level encryption on Supabase with CipherStash

supabase.com

21–30 of 54 posts

Re: Searchable field-level encryption on Supabase with CipherStash

#21
post #3

I’m having a hard time wrapping my head around what guarantees this does and does not make. If you can run “select * where secret_col == 10”… why does it matter that the column is encrypted?

CipherStash founder here. If a column is encrypted using standard encryption (like AES-GCM) then the values are non-deterministic and fully randomized. That means that if you encrypt the same value twice, you'll get 2 different ciphertexts. So the query: select * where secret_col == 10 Would actualy be: select * where secret_col == encrypt_aes(10); And values in secret_col will never match (because the output of encr…

Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users.

This is all started when I was the CTO of a health-tech and the engineers all had access to patient data. They needed DB access (data migrations, managing backups, reading non-sensitive fields) but not access to the sensitive values. 10m+ patient records - it made me nervous.

Row-level Security (RLS) was a nightmare, it was slow, easy to mess up and could be overridden by DBAs anyway. Plus the application connected to the DB using a service account so RLS was useless for end-user access control.

CipherStash solves the following:

* Hide sensitive data from direct database accessors (authorized or not!)

My original use-case. Direct access to the DB doesn't reveal sensitive data (unless the user also has permission to decrypt a specific record).

* Support or even replace application access controls

Decryption happens in the application layer and is based on end-user identity. That means a gap in application defenses is mitigated by the encryption. If the user can't decrypt the value then they can't access the data.

* Access auditing

Every sensitive data access is recorded (and by who) which is very useful for compliance and incident response. SQL query auditing is great for knowing what queries were run but it doesn't tell you what data was actually returned. Its also blind once data actually leaves the DB and can inadvertently leak sensitive data itself. Using key accesses with non-identifying value IDs to audit decryptions makes the auditing more reliable and super granular.

We have customers who use the tech to prove to their users (often large enterprise) that no internal staff can access the data.

* Agent access to data

The encryption shifts access control decisions down to the data layer on a per-value basis. This is incredibly useful for gating access to agents that need access to data. Every decryption requires authentication (e.g via Supabase Auth, Auth0 etc) and combining it with agent identities (creds issued to agents) means you can not only limit what agents can access, you can also audit exactly what _was_ accessed.

The searchable encryption is the enabler: value level encryption is what makes all of this possible. Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users.

Hope that helps! I'm so in the weeds I don't know if I explain things that well sometimes!

Re: Searchable field-level encryption on Supabase with CipherStash

#22

Earlier quoted context omitted.

CipherStash founder here. If a column is encrypted using standard encryption (like AES-GCM) then the values are non-deterministic and fully randomized. That means that if you encrypt the same value twice, you'll get 2 different ciphertexts. So the query: select * where secret_col == 10 Would actualy be: select * where secret_col == encrypt_aes(10); And values in secret_col will never match (because the output of encr…

the OP asked why, you more described the how. What is the benefit of all this?

I've replied above

Re: Searchable field-level encryption on Supabase with CipherStash

#23

Earlier quoted context omitted.

CipherStash founder here. If a column is encrypted using standard encryption (like AES-GCM) then the values are non-deterministic and fully randomized. That means that if you encrypt the same value twice, you'll get 2 different ciphertexts. So the query: select * where secret_col == 10 Would actualy be: select * where secret_col == encrypt_aes(10); And values in secret_col will never match (because the output of encr…

What does all this actually solve? Presumably SQL injection might still decrypt contents and order by/where clause enumeration would still be possible regardless. Keys must be stored in memory, on-disk or via a secret server meaning column encryption would not mitigate the impact of RCE/full shell compromise. Add in the cost of column level encryption when querying large volumes of data, this seems to be entirely ang…

See above but some additional points:

Keys are not stored in the database or in the application. Every data key is derived at query time via a 2-party system: 1. by the key server which manages root-key material (stored in an HSM or traditional KMS) 2. keys derived via a user credential at the time of the query

SQL Injection case: An adversary signs up to your app, pulls of a SQLi and you might think that the app will just decrypt the values. It won't because the adversary's auth cred can't derive keys to decrypt anything but the data they are explicitly allowed to access.

RCE/shell compromise: Because keys are derived on demand these kinds of attacks are largely inert. If the attacker could read memory from the running app on-demand then they could conceivably see credentials in flight but each key only ever decrypts one value so the blast radius is limited.

On the cost of decryption: Queries typically only return a subset of the values in a table and, importantly, don't need to decrypt anything at all. Querying and decryption are independent. If you DO need to decrypt a large volume of data, CipherStash can do it at 10,000 values per second.

Query performance is under 1ms for simple queries and no-more than 200-300ms for more complex queries.

Full benches here: https://github.com/cipherstash/benches

No security tool is perfect, nor "gold plated" but this tech meaningfully reduces blast area of attacks, makes many attacks inert and gives you an incredibly reliable audit trail.

Re: Searchable field-level encryption on Supabase with CipherStash

#24

Earlier quoted context omitted.

CipherStash founder here. If a column is encrypted using standard encryption (like AES-GCM) then the values are non-deterministic and fully randomized. That means that if you encrypt the same value twice, you'll get 2 different ciphertexts. So the query: select * where secret_col == 10 Would actualy be: select * where secret_col == encrypt_aes(10); And values in secret_col will never match (because the output of encr…

Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users. This is all started when I was the CTO of a health-tech and the engineers all had access to patient data. They needed DB access (data migrations, managing backups, reading non-sensitive fields) but not access to the sensitive values. 10m+ patient records - it made me nervous. Row-level Security (RLS) was a…

On auditing, is that CipherStash or the application doing the auditing? Seems like your logs would be huge if it contained values of queries

Re: Searchable field-level encryption on Supabase with CipherStash

#25

Earlier quoted context omitted.

What does all this actually solve? Presumably SQL injection might still decrypt contents and order by/where clause enumeration would still be possible regardless. Keys must be stored in memory, on-disk or via a secret server meaning column encryption would not mitigate the impact of RCE/full shell compromise. Add in the cost of column level encryption when querying large volumes of data, this seems to be entirely ang…

Careful, they’ll take “gold plated compliance security” and use it in sales pitches

If we do, I'll be sure to attribute it to this thread :p

Re: Searchable field-level encryption on Supabase with CipherStash

#26

Earlier quoted context omitted.

Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users. This is all started when I was the CTO of a health-tech and the engineers all had access to patient data. They needed DB access (data migrations, managing backups, reading non-sensitive fields) but not access to the sensitive values. 10m+ patient records - it made me nervous. Row-level Security (RLS) was a…

On auditing, is that CipherStash or the application doing the auditing? Seems like your logs would be huge if it contained values of queries

CipherStash - specifically the key service. There is a lot of data for sure but we only record an identifier for each value and (optionally) the user ID. It compresses well.

Re: Searchable field-level encryption on Supabase with CipherStash

#27

Earlier quoted context omitted.

On auditing, is that CipherStash or the application doing the auditing? Seems like your logs would be huge if it contained values of queries

CipherStash - specifically the key service. There is a lot of data for sure but we only record an identifier for each value and (optionally) the user ID. It compresses well.

So an identifier is kept for each column returned by a query?

If the value changes would you be able to get from the id you have audited back to the value that was returned at that point in time?

Re: Searchable field-level encryption on Supabase with CipherStash

#28

Earlier quoted context omitted.

CipherStash - specifically the key service. There is a lot of data for sure but we only record an identifier for each value and (optionally) the user ID. It compresses well.

So an identifier is kept for each column returned by a query? If the value changes would you be able to get from the id you have audited back to the value that was returned at that point in time?

Not just each column, each value.

The answer to your question is yes.

Explanation: The identifier is actually for the key that encrypts the value (1 unique key per value).

1. When the value is encrypted for the first time, it gets an ID. 2. When its decrypted, the application requests the key for that ID from the key-server (key-server records that the data was accessed)* 3. When updating, the same data key is used so the ID is persistent*

* Technically the key server doesn't return a key, it generates partial key material that can be used to derive the data key in the app. It can do this at up to 10,000 keys per second. * You can also tag each value with the table/column name and the row-id to link everything together

The 3 values form the "descriptor" of a value: `table/column/id`.

Re: Searchable field-level encryption on Supabase with CipherStash

#29

Earlier quoted context omitted.

So an identifier is kept for each column returned by a query? If the value changes would you be able to get from the id you have audited back to the value that was returned at that point in time?

Not just each column, each value . The answer to your question is yes. Explanation: The identifier is actually for the key that encrypts the value (1 unique key per value). 1. When the value is encrypted for the first time, it gets an ID. 2. When its decrypted, the application requests the key for that ID from the key-server (key-server records that the data was accessed)* 3. When updating, the same data key is used…

So the audit log contains an ID for a key which could decrypt a value - does the audit log also contain the encrypted value itself? If not, how do you go from the audit log back to the original value in 1?

Re: Searchable field-level encryption on Supabase with CipherStash

#30

Earlier quoted context omitted.

Not just each column, each value . The answer to your question is yes. Explanation: The identifier is actually for the key that encrypts the value (1 unique key per value). 1. When the value is encrypted for the first time, it gets an ID. 2. When its decrypted, the application requests the key for that ID from the key-server (key-server records that the data was accessed)* 3. When updating, the same data key is used…

So the audit log contains an ID for a key which could decrypt a value - does the audit log also contain the encrypted value itself? If not, how do you go from the audit log back to the original value in 1?

No, not by default. You could but as you said, that would be a A LOT of data.

It depends on your setup. If you're using Supabase, one way is to send the logs to Clickhouse and use the Clickhouse partner integration to query the audit logs and join it to the actual data.

Keeping only the ids in the audit log means you need to stitch the data together later. This means you don't accidentally leak data via your audit trail!

Post reply on HN