Live data from Hacker News

Searchable field-level encryption on Supabase with CipherStash

supabase.com

51–54 of 54 posts

Re: Searchable field-level encryption on Supabase with CipherStash

#51
post #3

I’m having a hard time wrapping my head around what guarantees this does and does not make. If you can run “select * where secret_col == 10”… why does it matter that the column is encrypted?

CipherStash founder here. If a column is encrypted using standard encryption (like AES-GCM) then the values are non-deterministic and fully randomized. That means that if you encrypt the same value twice, you'll get 2 different ciphertexts. So the query: select * where secret_col == 10 Would actualy be: select * where secret_col == encrypt_aes(10); And values in secret_col will never match (because the output of encr…

Thanks Dan, this is very interesting. From someone deeply involved in privacy I see the benefits of the auditability and risk mitigation in cases like SQL injection.

What does the risk profile look like in a full leak of the encrypted database?

If you had a column of integers are you able to order them without decrypting? Check that values are the same? Identify null values?

Re: Searchable field-level encryption on Supabase with CipherStash

#52
post #51

Earlier quoted context omitted.

CipherStash founder here. If a column is encrypted using standard encryption (like AES-GCM) then the values are non-deterministic and fully randomized. That means that if you encrypt the same value twice, you'll get 2 different ciphertexts. So the query: select * where secret_col == 10 Would actualy be: select * where secret_col == encrypt_aes(10); And values in secret_col will never match (because the output of encr…

Thanks Dan, this is very interesting. From someone deeply involved in privacy I see the benefits of the auditability and risk mitigation in cases like SQL injection. What does the risk profile look like in a full leak of the encrypted database? If you had a column of integers are you able to order them without decrypting? Check that values are the same? Identify null values?

> What does the risk profile look like in a full leak of the encrypted database?

We actually have a number of different schemes which trade leakage for performance/storage overhead/compatibility. Each one has its own type in Postgres so you just create a column with whatever type you want.

The weakest (most leakage) is OPE - order preserving encryption. We use an encoding scheme that works like scientific notation so an attacker with a DB snapshot would learn relative order but not the size of the gaps between values. Its major benefit is compatibility - it works everywhere.

The strongest is Block ORE - Order Revealing Encryption. A DB snapshot reveals nothing more than standard randomized encryption (IND-CPA2 semantic security). The tradeoff is that BlockORE values are much bigger: 32-bit integer goes to 384 bytes. On Postgres its still very fast and works with standard B-trees.

> If you had a column of integers are you able to order them without decrypting?

Yes. For either scheme, its just an EQL function:

SELECT * FROM foo ORDER by eql_v3.ord_term(x);

> Check that values are the same?

-- $1: encrypt(query) SELECT * FROM foo WHERE eql_v3.eq_term(x) = eql_v3.eq_term($1);

> Identify null values?

NULL is just like any value for OPE/ORE - encrypt it and use that to query -- $1: encrypt(NULL) SELECT * FROM foo WHERE eql_v3.eq_term(x) = eql_v3.eq_term($1);

NULL values can be encrypted with either scheme. Very safe to do so with the ORE scheme (fully randomized values). If encrypting strings, NULLs would reveal length but you can always pad if not leaking value length is important to your security model.

In the OPE scheme, NULL would encrypt deterministically so if you expect a lot of NULLs in your data this could leak the distribution.

We are working on a capability now called Leakage Tuned Domains. The goal here is not to eliminate leakage but scope it to a specific data type (e.g. birthdays) such that any leakage cannot be an advantage to an attacker. Think k-anonymisation or differential-privacy on steroids.

Some links if you want to learn more: https://github.com/cipherstash/encrypt-query-language https://cipherstash.com/blog/fixing-a-1-in-256-bug-in-cllw-o...

Re: Searchable field-level encryption on Supabase with CipherStash

#53

Earlier quoted context omitted.

No, not by default. You could but as you said, that would be a A LOT of data. It depends on your setup. If you're using Supabase, one way is to send the logs to Clickhouse and use the Clickhouse partner integration to query the audit logs and join it to the actual data. Keeping only the ids in the audit log means you need to stitch the data together later. This means you don't accidentally leak data via your audit tr…

Ok Im having trouble reconciling this comment > SQL query auditing is great for knowing what queries were run but it doesn't tell you what data was actually returned With what you’re saying - sounds like cipher doesnt tell you what data was actually returned either - it will only tell you if the user could have received a decrypted version of the data right?

Sorry I was imprecise.

Query logging would tell you what queries were executed. E.g the following would tell you that 10 user records were returned with name and email but not which users.

SELECT name, email FROM users LIMIT 10;

CipherStash can tell you the ID of every row that was returned, like:

users;email,name;1 users;email,name;2 users;email,name;3 etc

If you need to know the actual values instead of just the IDs (say when doing an investigation), you can do a query to join the audit data with the users table.

Assuming audit logs are in a table called audit with a column called record_id and a timestamp:

-- full forensic analysis SELECT name,email FROM audit LEFT JOIN users ON users.id = audit.record_id WHERE timestamp BETWEEN $1 AND $2;

And because the data is encrypted, that query will ALSO be audited. Not even admins escape the auditing :p

All of this is important for "materiality" investigations. If you suspect some data has been accessed inappropriately, knowing how much and exactly which records is very useful - especially if deciding if customer/regulator notification is required.

Re: Searchable field-level encryption on Supabase with CipherStash

#54
post #51

Earlier quoted context omitted.

Thanks Dan, this is very interesting. From someone deeply involved in privacy I see the benefits of the auditability and risk mitigation in cases like SQL injection. What does the risk profile look like in a full leak of the encrypted database? If you had a column of integers are you able to order them without decrypting? Check that values are the same? Identify null values?

> What does the risk profile look like in a full leak of the encrypted database? We actually have a number of different schemes which trade leakage for performance/storage overhead/compatibility. Each one has its own type in Postgres so you just create a column with whatever type you want. The weakest (most leakage) is OPE - order preserving encryption. We use an encoding scheme that works like scientific notation so…

Very cool Dan, thanks for the details!
Post reply on HN