Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Soft deletion probably isn't worth it

brandur.org

Post reply on HN
Print view View original

111–120 of 514 posts

Re: Soft deletion probably isn't worth it

#111

Post by Terr_ »

adrianmsmith wrote: post #99

> Instead of keeping deleted data in the same tables from which it was deleted from, there can be a new relation specifically for storing all deleted data The disadvantage of this is that if you ever do want to access this "deleted" data, e.g. in admin or compliance tools, you now have to do it in two different ways, one way for the main data and a different way in case the data has been "deleted". The article assert…

> The article asserts you'll never need to "undelete" the data.

IMO it's worth distinguishing between (A) some kind of "click to undelete" feature versus (B) simply having that old-data conveniently exposed for a developer to manually-edit things or craft database-change scripts.

In practice I've only ever seen the latter get used, because it requires a developer to figure out how the heck to get "the parts that matter" back while preserving the integrity of other newer data and obeying certain business-rules.

Re: Soft deletion probably isn't worth it

#112

Post by sam_lowry_ »

I do not understand the foreign keys issue. Do not use the deleted_at timestamp that is nullable by default. Instead, nullify the field when the line is deleted. Foreign keys on NULL values will be possible.

In any case, soft deletion is usually a sign of incompetence. Whenever I saw it on a project, both soft deletion and the project turned sour.

Re: Soft deletion probably isn't worth it

#113

Post by openthc »

I use a delta-log table, so each INSERT/UPDATE/DELETE on objects I care about are captured (via trigger) -- but that one has to get date partitioned. So in my system a DELETE statement (and DELETE CASCADE) work as expected -- any history has to be discovered from the logs

Re: Soft deletion probably isn't worth it

#114

Post by btown »

dfee wrote: post #25

My experience is that soft-deletes are blunt tools bridging the gap between hard deletes and event sourcing (capturing all the changes against the table, in a replay-worthy stream). Event sourcing is hard – because the engineers responsible for setting it up and managing it aren't generally well skilled in this domain (myself included) and there aren't a wealth of great tools helping engineers find their way into the…

The question for either of these systems IMO is: do you trust that a change from your upstream represents a true, everlasting intention, or is it something that may need to be reinterpreted or rolled back in the future?

At my startup, soft deletes for our SKUs are critical, because we work with data sources where notoriously both the technical systems and the humans driving will all-too-frequently accidentally represent something to our connection as deleted. Or there might be an irrecoverable error when asking "what things are still active upstream" - but that doesn't mean the SKUs are deleted, we might just not have certain live details until a bugfix is made. So "error status" and "soft delete" are somewhat synonymous, and both require investigation into root causes and root intents. Yes, the concept of "unerrored and active" is peppered through our codebase and analytics - but our ability to recover from supplier technical mistakes is much higher as a result. And we could absolutely do this with an event sourced system - but the tooling for relational databases is so much better, it's night and day.

Re: Soft deletion probably isn't worth it

#115

Post by jrockway »

nirvdrum wrote: post #95
giantg2 wrote: post #3

"The concept behind soft deletion is to make deletion safer, and reversible." That's one part. The other part is that in many industries you have regulatory data retention and audit requirements. This is arguably the most valuable and common reason to perform Logical deletes.

My experience at a few start-ups has been that account deletion just isn't prioritized. It's not a focus when building an MVP. If the application ever gains traction, everyone is then terrified they'll accidentally delete customer data that they never delete anything. It's a shame. As a user, when I delete my account or data in my account, I want you to permanently delete it, not keep it around and just make inaccess…

If it makes you feel better, the startups that don't have time to delete your data probably don't have a viable disaster recovery plan either.

As we learned from the Atlassian snafu, even giant companies with billions in revenue often can't recover from disasters. (I try to test mine every 6 months. I've never had a test go perfectly.)

Re: Soft deletion probably isn't worth it

#116

Post by marcosdumay »

justin_oaks wrote: post #30

The author didn't mention it, but restoring data from a database backup is a perfectly reasonable way to handle undeletes. By this I mean the situations that are "Oh crap, we didn't mean to delete that!" instead of usual business operations. I've probably restored data from backup maybe 4 times in my career. I greatly prefer to do this on the rare one-off scenario than to deal with the overhead of soft deleting every…

The difference in framing one gets by looking around is amazing, even funny.

> I've probably restored data from backup maybe 4 times in my career.

Yet, I often use soft-deletes because it allows people to undelete things from the software interface and not call me all day long.

But that's not the most common reason I have for them. Normally it is because the data just can not be gone, and the full table is still important somewhere.

Re: Soft deletion probably isn't worth it

#117

Post by munk-a »

yladiz wrote: post #96
munk-a wrote: post #84

Earlier quoted context omitted.

This is a really scenario specific question - sometimes it's needed, sometimes it isn't. At my shop we have customers that will suspend their account but our sales team is pretty damn awesome so usually they end up renewing after going a while without our product - so being able to easily restore a large swath of former customers with all their permissions and preferences intact with a simple click of a button is a h…

If I was a tech lead and also wanted to advocate for hard deletion, I would ask the question for this scenario: "What's the cost of keeping all this data unnecessarily, modifying most queries to filter for deleted data, and dealing with other various consequences of soft deletion, and how does this cost compare with the cost of building a bespoke tool to restore deleted data at a customer's request within a certain t…

As your data architect I'd probably answer your question "Well, it'll take a little while longer to vet all the indices - since we'll probably want most indices to filter on WHERE deleted_at IS NULL but we'll likely want a few without that constraint for managing undeletion. We'll use more space on disk which, honestly, is pretty much a non-issue in the modern world - and if it gets bad enough we can always partition the table on deletion status and dump the soft deleted rows on a secondary server... but I wouldn't worry about that until we hit facebook scale. In terms of application developer time - I'll probably need an hour of their time to explain it once and all existing queries keep working as they're working now... and as for that undelete tool, well, we don't actually have to build it - if we don't build it we can just ask devs to submit manual queries to undelete data rows as needed via our migration, or oneoff or console interface. We'll want to do a sweep for any queries that reference tables dependent on the soft-delete having table - just to make sure they're throwing an INNER JOIN against the view but, honestly, we could just bop those bugs on the head if they ever come up."

Like, I'm definitely not saying soft-deletion is always the answer, it takes additional effort and planning, but it's really, really easy to do safely.

Re: Soft deletion probably isn't worth it

#118

Post by layer8 »

adrianmsmith wrote: post #99

> Instead of keeping deleted data in the same tables from which it was deleted from, there can be a new relation specifically for storing all deleted data The disadvantage of this is that if you ever do want to access this "deleted" data, e.g. in admin or compliance tools, you now have to do it in two different ways, one way for the main data and a different way in case the data has been "deleted". The article assert…

> now have to do it in two different ways

Use a view.

> if there are any unique constraints e.g. on username or email address

Have those in a dedicated table where they aren’t deleted, and add a synthetic key referenced by the other tables.

Re: Soft deletion probably isn't worth it

#119

Post by vbezhenar »

dfee wrote: post #25

My experience is that soft-deletes are blunt tools bridging the gap between hard deletes and event sourcing (capturing all the changes against the table, in a replay-worthy stream). Event sourcing is hard – because the engineers responsible for setting it up and managing it aren't generally well skilled in this domain (myself included) and there aren't a wealth of great tools helping engineers find their way into the…

You can restore to any point of time from your database backup. So it can cover some requirements.

Re: Soft deletion probably isn't worth it

#120

Post by Pxtl »

Trivial case I hit:

1) Client wants to remove user from the system who have left their org but

2) There are objects that were contributed by that user which are required to persist beyond the user's deletion.

Those are ideal cases for soft deletion. We can still query information about the deleted user to explain who created this object, with the note that their account has been deleted.

Probably I should be doing full event-sourcing for this case, but delete flag works well. MS offers temporal tables for this use case and I'm still considering the implications there -- AFAIK ORM support is WIP.

And unlike the article author, I have used soft deletion to undelete things. Many times. Maybe he has better users than I do, I don't know.

Post reply on HN