Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Soft deletion probably isn't worth it

brandur.org

Post reply on HN
Print view View original

301–310 of 514 posts

Re: Soft deletion probably isn't worth it

#301

Post by bpicolo »

twelve40 wrote: post #268
alerighi wrote: post #216

Earlier quoted context omitted.

It may be convenient, but under the GDPR is illegal. When an user deletes an account, all the personal data associated with that user must be deleted (or anonymize it in a way that it's no longer possible to associate it back to the particular user). You cannot just keep user information forever "just in case" they are useful again.

You can most definitely keep around the non-PII data you've generated for that user in your app, such as synthetic user ID's and other stuff. Furthermore, for some apps (fintech) in some countries (US) you are actually obligated to keep around certain information for several years. It really doesn't have to be all or nothing.

Insurance has to keep around info for the lifetime of the policy. For some products, that's decades.

Re: Soft deletion probably isn't worth it

#302

Post by EUROCARE »

avianlyric wrote: post #277
lazide wrote: post #274

Earlier quoted context omitted.

This should have the asterisk of * after the period of time you’ve said it will take for it to be deleted. It’s almost impossible to guarantee instant delete, folks just care if it’s gone in a documented and/or reasonable amount of time, and predictably. Correct?

Sure, but that time period is usually limited to 30 days, with an additional 30 day extension allowed if you contact the person and tell them you need more time. You can’t just declare that it take years to delete data, GDPR sets reasonable limits on how long a company can delay true deletion.

Even if the company just issued you an invoice, that makes it obligated by law to hold onto your PII for the next 5 years - and GDPR of course has an exception for that.

Re: Soft deletion probably isn't worth it

#303

Post by rodw »

djur wrote: post #296
rodw wrote: post #290

Earlier quoted context omitted.

I can see that use case and I guess if that's a popular idiom within a particular dev community it's not indefensible, but from a first-principles code-complete/pragmatic-programmer kind of lens, that seems to me like a really terrible naming convention if that's your intent. It seems to me that having a date-type column named `is_frobbed` is highly preferable to `frobbed_at` if your intent is for `IS NOT NULL` to me…

I would use "deleted_after" for the use case you describe (scheduled deletions). "deleted_at" is saying that an event occurred (deletion) at a specific time. There is never going to be a case where you have a deletion time for a record that is not deleted, nor should there ever be a time where a record is deleted but a time is not recorded. So a single timestamp column is a parsimonious solution.

I agree `deleted_after` is a more appropriate name than `deleted_at`, but it starts to run into naming conventions again. For example `published_at` is often used in editorial systems (newspapers, blogs, etc.) for things that "embargoed until" some future date. `published_after` or even just `pub_dt` would probably be more appropriate, but I feel like the `_at` suffix is well established in some contexts.

For what it's worth, I have built and managed systems where "delete/disable this thing in the future" is a valid use case, but I'm wondering whether/how often/how the date aspect of the Rails-style `deleted_at` is actually used. Does anyone ever care when the record was deleted, or is it just extra metadata that is occasionally used in an ad hoc way for debugging or diagnostics. If the typical rails app replaced `deleted_at` with boolean-valued column, would it actually matter?

Re: Soft deletion probably isn't worth it

#304

Post by jwarden »

robertlagrant wrote: post #240
efsavage wrote: post #193

Earlier quoted context omitted.

Also if people know that deletion is reversible, they're more likely to actually do it, which can keep things generally tidier. I don't actually like using a "deleted" column, my standard table has a status column, and deleted is one of those states, along with active/pending/suspended/etc, as the needs dictate. This way I get soft deletes for basically free both in the schema, but also in the queries (which would ge…

That still has the same issues. You have to remember to set every linked table's records to the same state, or remember to query every linked table through the table that has the lifecycle column on it.

Forgetting to include the lifecycle column in your query may mean not understanding what question you are trying to answer with your query.

Re: Soft deletion probably isn't worth it

#305

Post by mlyle »

michaelt wrote: post #228
dspillett wrote: post #197

Earlier quoted context omitted.

> Since I can no longer rely on ON DELETE CASCADE relationships Cascaded deletes scare me anyway. It only takes one idiot to implement UPSERT as DELETE+INSERT because it seems easier, and child data is lost. You could always use triggers to cascade you soft-delete flags as an alternative method, though that would be less efficient (and more likely to be buggy) than the built-in solution that cascaded deletes are. If…

> It only takes one idiot to implement UPSERT as DELETE+INSERT because it seems easier, and child data is lost. Seems unfortunate to miss out on all the referential integrity benefits of a serious database when hiring standards, training and code reviews should all be preventing idiotic changes. If I’m making a shopping cart system, I want to know every order line belongs to an order, every order belongs to a user an…

> Seems unfortunate to miss out on all the referential integrity benefits of a serious database when hiring standards, training and code reviews should all be preventing idiotic changes.

If you don't use ON DELETE CASCADE, the actual foreign key constraint gives you a meaningful error-- that you need to delete some stuff to have referential integrity.

ON DELETE CASCADE --- you're telling it "eh, if you need to delete some stuff to avoid an error, go ahead, don't bother me, do what I mean".

Re: Soft deletion probably isn't worth it

#306

Post by rtpg »

I believe you can get most of the advantages of soft deletion through a notion of archival.

Usually archival does the main thing (“get this out of my main resource list”) without breaking audit trails or resource links. For people for whom this is insufficient, you can of course offer hard deletion.

Re: Soft deletion probably isn't worth it

#307

Post by femiagbabiaka »

JohnBooty wrote: post #151

I've been a software dev since the 90s and at this point, I've learned to basically do things like audit trails and soft deletion by default, unless there's some reason not to. Somebody always wants to undelete something, or examine it to see why it was deleted, or see who changed something, or blah blah blah. It helps the business, it helps you as developer by giving you debug information as well as helping you to c…

Having multiple use cases for data is normal and okay. Treating your primary data store that is designed for one set of use cases and using it for them all is very very bad, even before you hit scale. Know which data store to use and when for a use case is a super power than can allow you to scale to much higher magnitudes than one would intuit.

Re: Soft deletion probably isn't worth it

#308

Post by [deleted] »

brtkdotse wrote: post #13
giantg2 wrote: post #3

"The concept behind soft deletion is to make deletion safer, and reversible." That's one part. The other part is that in many industries you have regulatory data retention and audit requirements. This is arguably the most valuable and common reason to perform Logical deletes.

In banking and bookkeeping, there’s no such thing as a “delete”. Once something is in the ledger you can’t undo it - you have to make a new entry that negates the old one.

[deleted]

Re: Soft deletion probably isn't worth it

#309

Post by d0100 »

rodw wrote: post #275
madisp wrote: post #229

Earlier quoted context omitted.

I'd argue that the simple `deleted_at IS NULL` check is not broken - unless your product / domain specifically allows and requires scheduled future deletions adding such logic can easily introduce bugs. For example, you could to get the comparison flipped by accident, and if it's only in one place out of many that bug could go unnoticed for a while.

If any non-null value for deleted_at indicates a logical delete, it seems like TIMESTAMP is the wrong data type for that column. I mean I guess I can imagine a scenario where you want to know _when_ a record was logically deleted in addition to a flag that says is_deleted, but maybe that should be two different columns. Or at the very least, `deleted_at` is a misleading name in that case. Interpreting `deleted_at` as…

> If any non-null value for deleted_at indicates a logical delete, it seems like TIMESTAMP is the wrong data type for that column.

Not if I also want the timestamp of when it was deleted

Re: Soft deletion probably isn't worth it

#310

Post by cpursley »

JohnBooty wrote: post #151

I've been a software dev since the 90s and at this point, I've learned to basically do things like audit trails and soft deletion by default, unless there's some reason not to. Somebody always wants to undelete something, or examine it to see why it was deleted, or see who changed something, or blah blah blah. It helps the business, it helps you as developer by giving you debug information as well as helping you to c…

Can you recommend any best practices, design patterns, etc - for implementing soft delete in SQL systems?
Post reply on HN