Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Soft deletion probably isn't worth it

brandur.org

Post reply on HN
Print view View original

241–250 of 514 posts

Re: Soft deletion probably isn't worth it

#241

Post by marcofiset »

tsuujin wrote: post #234
augustl wrote: post #164

Earlier quoted context omitted.

This is why I don't understand why Datomic isn't more popular. Pretty much every system I've worked on never needed to scale past 100s of writes per second due to hard limits on the system (internal backoffice stuff, fundamenally scoped/shardable to defined regions, etc etc). And since Datomic is built with that in mind, you get the trade-off of full history, first class transactions and being able to query for thing…

It’s hard to get adoption for expensive toys. I think Datomic is neat, and I’d like to use it, but it is prohibitively expensive for a personal or hobby project. Personal projects are where I get excited about tech, and when I’m excited I’m more likely to adopt it in my day job. They’re really shooting themselves in the foot by not having a one-click install free tier or a self hosted option.

You may want to look into XTDB then. Not quite the same as Datomic, but they share many similarities. It's free and open-source.

Re: Soft deletion probably isn't worth it

#242

Post by robertlagrant »

wst_ wrote: post #236
wnevets wrote: post #182

Earlier quoted context omitted.

This is something that I was forced to learn the hard way more than once. Literally today I needed to undelete a record because a customer was confused by what the "delete" button did and wanted their record back.

Isn't it the problem of UI, though. If the user would be informed about the consequences (possibly with bold red font and with a confirmation checkbox) would they still click that button?

One will, one day.

Re: Soft deletion probably isn't worth it

#243

Post by smolder »

alerighi wrote: post #216
gmiller123456 wrote: post #172

Earlier quoted context omitted.

The author uses the "no one ever undeleted anything" as the primary justification. I think this is the part they miss. I've never undeleted a user either, but there have been many times I've gone back to look at something. Either a complaint finally gets around to me as to why the user wanted their account deleted (e.g. feature not working) and it helps to figure out why. Or they're returning and want things set up l…

It may be convenient, but under the GDPR is illegal. When an user deletes an account, all the personal data associated with that user must be deleted (or anonymize it in a way that it's no longer possible to associate it back to the particular user). You cannot just keep user information forever "just in case" they are useful again.

I wonder what percentage of companies who even appear to comply with deletion requests actually do full deletion in practice. I suspect it's small, knowing how many things are coded to fake-delete for convenience. Businesses also tend to keep cold data backups around. (Maybe backups are exempted? I don't know.) There might even be cases where ostensibly deleted data can still be recovered from a disk, if they haven't overwritten.

Re: Soft deletion probably isn't worth it

#244

Post by nkozyra »

wvenable wrote: post #143
pbardea wrote: post #49

Earlier quoted context omitted.

> assuming the deletes are done appropriately This is one gripe I have with soft-deletion. Since I can no longer rely on ON DELETE CASCADE relationships, I need to re-defined these relationship between objects at the application layer. This gets more and more difficult as relationships between objects increase. If the goal is to keep a history of all records for compliance reasons or "just in case", I tend to prefer…

Often you don't have to rely on ON DELETE CASCADE relationships. Because you are never deleting anything, you will never have any orphaned records. If you don't want to see say Invoices for a deleted Customer then that's just another filter feature. Mostly I use soft-delete because for auditing requirements we pretty much can't remove anything but also because nothing ever truly goes away. If we have an Invoice or Or…

> Often you don't have to rely on ON DELETE CASCADE relationships. Because you are never deleting anything, you will never have any orphaned records

Exactly. Unless you're doing something silly like adding deleted at to bridge tables ... which, you probably don't need even in 1:many.

Re: Soft deletion probably isn't worth it

#245

Post by tsuujin »

marcofiset wrote: post #241
tsuujin wrote: post #234

Earlier quoted context omitted.

It’s hard to get adoption for expensive toys. I think Datomic is neat, and I’d like to use it, but it is prohibitively expensive for a personal or hobby project. Personal projects are where I get excited about tech, and when I’m excited I’m more likely to adopt it in my day job. They’re really shooting themselves in the foot by not having a one-click install free tier or a self hosted option.

You may want to look into XTDB then. Not quite the same as Datomic, but they share many similarities. It's free and open-source.

Neat, thanks!

Re: Soft deletion probably isn't worth it

#246

Post by joshuanapoli »

Atlassian's deletion-related outage demonstrates why soft deletion should be the default. Use hard deletion after a grace period for data that truly needs to be expunged. Even if undelete is not part of the normal workflow, experience shows that swift recovery from bugs and operator errors is a universal part of serving users. The less data motion involved in deletion (and recovery) the better for both the original deletion process and also any recovery process.

https://news.ycombinator.com/item?id=31015813

Re: Soft deletion probably isn't worth it

#247

Post by Akronymus »

gmiller123456 wrote: post #172
JohnBooty wrote: post #151

I've been a software dev since the 90s and at this point, I've learned to basically do things like audit trails and soft deletion by default, unless there's some reason not to. Somebody always wants to undelete something, or examine it to see why it was deleted, or see who changed something, or blah blah blah. It helps the business, it helps you as developer by giving you debug information as well as helping you to c…

The author uses the "no one ever undeleted anything" as the primary justification. I think this is the part they miss. I've never undeleted a user either, but there have been many times I've gone back to look at something. Either a complaint finally gets around to me as to why the user wanted their account deleted (e.g. feature not working) and it helps to figure out why. Or they're returning and want things set up l…

I messed up a mass update query enough times to leave myself SOME provision to undo it.

The exceptions are when there is a well tested query that affects a single account or something. Like GDPR

Re: Soft deletion probably isn't worth it

#248

Post by dataflow »

alerighi wrote: post #216
gmiller123456 wrote: post #172

Earlier quoted context omitted.

The author uses the "no one ever undeleted anything" as the primary justification. I think this is the part they miss. I've never undeleted a user either, but there have been many times I've gone back to look at something. Either a complaint finally gets around to me as to why the user wanted their account deleted (e.g. feature not working) and it helps to figure out why. Or they're returning and want things set up l…

It may be convenient, but under the GDPR is illegal. When an user deletes an account, all the personal data associated with that user must be deleted (or anonymize it in a way that it's no longer possible to associate it back to the particular user). You cannot just keep user information forever "just in case" they are useful again.

It's illegal but companies don't necessarily care to avoid soft deletes regardless. I think companies wait to get sued so they can try to argue in court why their soft deletions are reasonable and why it's too technically difficult for them to do hard deletes.

To be honest, in the age of modern overprovisioned storage drives that remap blocks frequently, I'm not really sure you can implement genuine "hard" deletes without choosing significantly unorthodox hardware (or destroying a drive every time you need a single bit erased), no matter how much you want to in software. One of those details that I'm both surprised and unsurprised doesn't seem to have been addressed legally. I feel like a court ought to at least buy this aspect of the argument, so maybe they'll buy that it can be difficult in terms of the software too? Who knows. My guess is that a reasonable court would accommodate something that's reasonable for a given company, but there are lots of variations that could fall into that category.

Re: Soft deletion probably isn't worth it

#249

Post by wnevets »

wst_ wrote: post #236
wnevets wrote: post #182

Earlier quoted context omitted.

This is something that I was forced to learn the hard way more than once. Literally today I needed to undelete a record because a customer was confused by what the "delete" button did and wanted their record back.

Isn't it the problem of UI, though. If the user would be informed about the consequences (possibly with bold red font and with a confirmation checkbox) would they still click that button?

All of that exist.

Re: Soft deletion probably isn't worth it

#250

Post by rubyist5eva »

One thing that I could find in the article: performance.

At least for our use case, soft deletes made everything slower because it's much harder to index. For our database we basically had to do an audit of all of our WHERE clauses and create partial indexes on "not yet deleted" records. Of course, this bloats your indexes/disk and hurts write performance so it's not a silver bullet.

We've also taken to inserting into "delete records tables" for records we may want to recover or for historical reasons. You still lose foreign keys but indexing and query optimization is a lot easier, and your old data is just still a simple query away.

Post reply on HN