Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Immutable Data (2015)

kevinmahoney.co.uk

Post reply on HN
Print view View original

11–20 of 43 posts

Re: Immutable Data (2015)

#11

Post by misterS »

kaba0 wrote: post #10
pil0u wrote: post #3

From a user's perspective, I can see a privacy drawback as well. Suppose that instead of a typical User table, you have a User_Revision table like suggested. Every time a user updates their account settings, you INSERT a new row there. If a user changes their email address, you get a row each time they update it. Not only the company gets an history of email addresses, but also they are tied to each other. If this in…

Slightly unrelated: A great way to solve this specific email issue is to store hashes for emails only, similarly to passwords. When a user needs a password reset, they should send an email to a specific address, the sender mail is hashed, and if it matches a user, a password reset link is replied. That way a data breach can’t expose user emails. (This is what Tildes use: https://tildes.net/settings/account_recovery ,…

This works great for password resets.

It does not work if you actually want to send emails, for instance notifications.

Re: Immutable Data (2015)

#12

Post by kaba0 »

misterS wrote: post #11
kaba0 wrote: post #10

Earlier quoted context omitted.

Slightly unrelated: A great way to solve this specific email issue is to store hashes for emails only, similarly to passwords. When a user needs a password reset, they should send an email to a specific address, the sender mail is hashed, and if it matches a user, a password reset link is replied. That way a data breach can’t expose user emails. (This is what Tildes use: https://tildes.net/settings/account_recovery ,…

This works great for password resets. It does not work if you actually want to send emails, for instance notifications.

That’s true, though many websites would be much better off if they would properly evaluate the question of do they actually need to send those?

Re: Immutable Data (2015)

#13

Post by vbezhenar »

kaba0 wrote: post #10
pil0u wrote: post #3

From a user's perspective, I can see a privacy drawback as well. Suppose that instead of a typical User table, you have a User_Revision table like suggested. Every time a user updates their account settings, you INSERT a new row there. If a user changes their email address, you get a row each time they update it. Not only the company gets an history of email addresses, but also they are tied to each other. If this in…

Slightly unrelated: A great way to solve this specific email issue is to store hashes for emails only, similarly to passwords. When a user needs a password reset, they should send an email to a specific address, the sender mail is hashed, and if it matches a user, a password reset link is replied. That way a data breach can’t expose user emails. (This is what Tildes use: https://tildes.net/settings/account_recovery ,…

While it's better than storing emails in clear-text, nobody uses vvkcbrxrsewf@gmail.com. Brute-forcing emails is going to be pretty effective. And if you have another e-mail database (which are plenty available in the open wild), I bet that success rate will be huge.

I like Apple's approach to generating random throwaway emails.

Re: Immutable Data (2015)

#14

Post by razzm256 »

pil0u wrote: post #3

From a user's perspective, I can see a privacy drawback as well. Suppose that instead of a typical User table, you have a User_Revision table like suggested. Every time a user updates their account settings, you INSERT a new row there. If a user changes their email address, you get a row each time they update it. Not only the company gets an history of email addresses, but also they are tied to each other. If this in…

Forgetting is a separate operation that can be (relatively) easily added whereas updating existing system to work w/ immutable data is an effort on a whole another scale.

Re: Immutable Data (2015)

#15

Post by kaba0 »

vbezhenar wrote: post #13
kaba0 wrote: post #10

Earlier quoted context omitted.

Slightly unrelated: A great way to solve this specific email issue is to store hashes for emails only, similarly to passwords. When a user needs a password reset, they should send an email to a specific address, the sender mail is hashed, and if it matches a user, a password reset link is replied. That way a data breach can’t expose user emails. (This is what Tildes use: https://tildes.net/settings/account_recovery ,…

While it's better than storing emails in clear-text, nobody uses vvkcbrxrsewf@gmail.com. Brute-forcing emails is going to be pretty effective. And if you have another e-mail database (which are plenty available in the open wild), I bet that success rate will be huge. I like Apple's approach to generating random throwaway emails.

You don’t have to use the same salt for each email, right? You can hash each email with a separate, random salt and “brute force” upon password recovery. That way while individual emails can still be restored, you can’t easily get a database of emails to mass-spam, which is the point (emails are not really private completely, anyway).

I also like apple’s method, but it is not either-or.

Re: Immutable Data (2015)

#16

Post by roenxi »

I'll argue that this is bad design. It works as long as the amount of data is small, but even then taking a low-data scenario and building lots of views or triggers just seems a bit weird. When I do that, it works for a month and then falls over in maintenance because you've got a table-based database where none of the important data is in a table?! This is not a design that will be flexible if needs change even slightly.

Immutable design is extremely powerful, but it needs to be a first-class citizen to get full benefit. Clojure's data structures are a great study in this - they squeeze a shocking amount of efficiency out because they have guarantees that the underlying data is immutable (eg, copy-and-slightly-update a large object is effectively a free operation, we have old & new objects available for comparison and that is lovely). Mimicking the same style of programming in, say, Java would gain none of the performance advantages or the logical conveniences. I expect it would be an uncomfortable programmer experience.

Here I think it would be more effective to design the tables as-usual and keep a log of all the changes separately. There is a chance the log will get out of sync with the active tables but frankly if that is a problem go use something designed with immutability in mind and don't twist PostgreSQL into pretzel shapes.

Re: Immutable Data (2015)

#17

Post by zambal »

steve_adams_86 wrote: post #6
pharmakom wrote: post #2

It’s a mistake to use timestamp for sorting versions instead of revision_id imo

What case would you make for using revision_id?

I'm not OP, but I can at least give an argument against using a timestamp for sorting versions: for various reasons system time is not guaranteed to be monotonic, which can result in cases where newer versions would be sorted before older versions.

See https://www.erlang.org/doc/apps/erts/time_correction.html#in... for more info. It's about time handling in the Erlang run-time system, but the issue it describes is universal.

And having an Erlang background, I would say timestamps are also tricky in the context of distributed systems.

Re: Immutable Data (2015)

#18

Post by bloak »

kaba0 wrote: post #10
pil0u wrote: post #3

From a user's perspective, I can see a privacy drawback as well. Suppose that instead of a typical User table, you have a User_Revision table like suggested. Every time a user updates their account settings, you INSERT a new row there. If a user changes their email address, you get a row each time they update it. Not only the company gets an history of email addresses, but also they are tied to each other. If this in…

Slightly unrelated: A great way to solve this specific email issue is to store hashes for emails only, similarly to passwords. When a user needs a password reset, they should send an email to a specific address, the sender mail is hashed, and if it matches a user, a password reset link is replied. That way a data breach can’t expose user emails. (This is what Tildes use: https://tildes.net/settings/account_recovery ,…

This is probably worth doing, but one should be aware that a typical e-mail address does not contain enough entropy to not be guessable offline.

Perhaps someone should deliberately leek a bogus database that contains the bcrypt hash of davey.jones.490@well-known-email-provider.com and see if that account starts receiving spam ... probably it won't.

There is (almost certainly) no associated e-mail account, but the following is the SHA-256 hash of firstname.lastname.threedigits, something like "davey.jones.490". I'm half-expecting someone to reply with the solution, though I think there are about 36 bits of entropy in my choice:

b8b6395265714064afb525f37e9969a7dbf09a4a2afa28812e917580be2dc667

Re: Immutable Data (2015)

#19

Post by jerrygenser »

roenxi wrote: post #16

I'll argue that this is bad design. It works as long as the amount of data is small, but even then taking a low-data scenario and building lots of views or triggers just seems a bit weird. When I do that, it works for a month and then falls over in maintenance because you've got a table-based database where none of the important data is in a table?! This is not a design that will be flexible if needs change even slig…

It's ironic that postgres the underlying storage model for postgres are immutable tables that marks records as "invisible" and asynchronously cleaned up.

Re: Immutable Data (2015)

#20

Post by roenxi »

jerrygenser wrote: post #19
roenxi wrote: post #16

I'll argue that this is bad design. It works as long as the amount of data is small, but even then taking a low-data scenario and building lots of views or triggers just seems a bit weird. When I do that, it works for a month and then falls over in maintenance because you've got a table-based database where none of the important data is in a table?! This is not a design that will be flexible if needs change even slig…

It's ironic that postgres the underlying storage model for postgres are immutable tables that marks records as "invisible" and asynchronously cleaned up.

Yeah. Immutable design is better, what more is there to say?

Unfortunately Postgres doesn't have an API to get at those well designed internals. It implements SQL.

Post reply on HN