Live data from Hacker News

YAGRI: You are gonna read it

scottantipa.com

31–40 of 161 posts

Re: YAGRI: You are gonna read it

#31

Just curious, how do people feel about this general style of soft deletes currently? Do people still use these in production or prefer to just delete fully or alternatively move deleted rows to a separate tables / schema? I find the complexity to still feel awkward enough that makes me wonder if deleted_at is worth it. Maybe there are better patterns out there to make this cleaner like triggers to prevent deletion, s…

> Maybe there are better patterns out there to make this cleaner

SQL:2011 temporal tables are worth a look.

Re: YAGRI: You are gonna read it

#32
post #24
post #18

Earlier quoted context omitted.

If you have a good audit log, it really doesn't matter. You can always restore it if need be. If you have no audit log(or a bad one), like lots of apps, then you have to care a lot. Personally, I just implement a good audit log and then I just delete with impunity. Worst case scenario, someone(maybe even me) made a mistake and I have to run undo_log_audit() with the id of the audit log entry I want to put back. Nearl…

Can you share more about what makes a good audit log? My company doesn’t currently have one and I’m a little lost on where to start. Should this be at the application code level, or the ORM, or the database itself?

Probably application level in most cases as those other levels probably don’t have all the information you want to include.

Re: YAGRI: You are gonna read it

#33
post #4

One thing I do quite frequently which is related to this (and possibly is a pattern in rails) is to use times in place of Booleans. So is_deleted would contain a timestamp to represent the deleted_at time for example. This means you can store more information for a small marginal cost. It helps that rails will automatically let you use it as a Boolean and will interpret a timestamp as true.

This is all well and good until you need to represent something that happened on Jan 1 1970 00:00 UTC.

32-bit UNIX timestamps are often signed so you can actually go before that, but most UNIX timestamps are 64-bit now, which can represent quite a larger range. And SQL datetime types might have a totally different range.

Not that it really matters; deleted_at times for your database records will rarely predate the existence of said database.

Re: YAGRI: You are gonna read it

#34
post #4

One thing I do quite frequently which is related to this (and possibly is a pattern in rails) is to use times in place of Booleans. So is_deleted would contain a timestamp to represent the deleted_at time for example. This means you can store more information for a small marginal cost. It helps that rails will automatically let you use it as a Boolean and will interpret a timestamp as true.

This is all well and good until you need to represent something that happened on Jan 1 1970 00:00 UTC.

Leave it null for non-deleted items.

Re: YAGRI: You are gonna read it

#35

Just curious, how do people feel about this general style of soft deletes currently? Do people still use these in production or prefer to just delete fully or alternatively move deleted rows to a separate tables / schema? I find the complexity to still feel awkward enough that makes me wonder if deleted_at is worth it. Maybe there are better patterns out there to make this cleaner like triggers to prevent deletion, s…

Financial world: records have a "close" or "expire" date which is then purged after some period of time. A deletion doesn't just happen, the record is updated to be "closed" or "expired" and some time after that it's deleted.

Something like a loan could live in a production environment for well over a year after closing, while an internal note may last just a month.

Re: YAGRI: You are gonna read it

#37
post #24
post #18

Earlier quoted context omitted.

If you have a good audit log, it really doesn't matter. You can always restore it if need be. If you have no audit log(or a bad one), like lots of apps, then you have to care a lot. Personally, I just implement a good audit log and then I just delete with impunity. Worst case scenario, someone(maybe even me) made a mistake and I have to run undo_log_audit() with the id of the audit log entry I want to put back. Nearl…

Can you share more about what makes a good audit log? My company doesn’t currently have one and I’m a little lost on where to start. Should this be at the application code level, or the ORM, or the database itself?

That depends on where the data you need to keep track of is and your architecture. The important thing is, you want your audit log to be able to tell you:

  * Who
  * What
  * When
  * Ideally Why
For any change in the system. Also when storing the audit log, take into account that you might need to undo things that happened(not just deletes). For instance maybe some process went haywire and inserted 100k records it wasn't supposed to. A good audit log, you should be able to run something like undo_log_audit(rec1, rec100k) and it will do the right thing. I'm not saying that code needs to exist day 1, but you should take into account the ability to do that when designing it.

Also you need to take into account your regulatory environment. Sometimes it's very very important that your audit logs are write once, and read only afterwards and are stored off machine, etc. Other times it's just for internal use and you can be a little more lax about date integrity of your audit logs.

Our app is heavily database centric. We push into the DB the current unix user, the current PID of the process connecting to the DB, etc(also every user has their own login to the DB so it handles our authentication too). This means our database(Postgres) does all of the audit logging for us. There are plenty of Postgres audit logging extensions. We run 2 of them. One that is trigger based creating entries in a log_audit table(which the undo_log_audit() code uses along with most reporting use cases) and a second one that writes out to syslog(so we can move logs off machine and keep them read only). We are in a regulated industry that gets audited regularly however. Not everyone needs the same level of audit logging.

You need to figure out how you can answer the above questions given your architecture. Normally the "Why" question is hard to answer without talking with a human, but unless you have the who, what and when, it's nearly impossible to even get to the Why part of the question.

Re: YAGRI: You are gonna read it

#38
post #36

To every point there is a counterpoint. So what?

Well in the same vain that we discuss "points" and talk about the merits, its useful to discuss and understand their counter points. I for one did not know about this and thought it was insightful when building a product that hasn't fully been scoped out and is more greenfield

Re: YAGRI: You are gonna read it

#40
post #33

Earlier quoted context omitted.

This is all well and good until you need to represent something that happened on Jan 1 1970 00:00 UTC.

32-bit UNIX timestamps are often signed so you can actually go before that, but most UNIX timestamps are 64-bit now, which can represent quite a larger range. And SQL datetime types might have a totally different range. Not that it really matters; deleted_at times for your database records will rarely predate the existence of said database.

It's not about the scale, it's that `if (0)` will evaluate to `false` in many languages.
Post reply on HN