Ask HN: What's your preferred way of ensuring complex database integrity?
11–20 of 58 posts
Re: Ask HN: What's your preferred way of ensuring complex database integrity?
#12I've opted to build an "API" within the database consisting of views for read-access and stored procedures that can validate relationships upon insert/update. It's pretty clean so far as it allows the data needed for integrity checks to remain in the database.
This hasn't made it to production yet, so take it with a grain of salt :)
Re: Ask HN: What's your preferred way of ensuring complex database integrity?
#13The main reason is change. As I evolve the code base, I might want to do something today which is principled, but seemed unprincipled yesterday. Database constraints have tended to hit me the hardest at the worst times too, e.g. being unable to make an emergency fix in prod. Or prod can't come up because the flyway script (which succeeded in test/stage) got snagged on a constraint when it ran on prod data.
Re: Ask HN: What's your preferred way of ensuring complex database integrity?
#14Unfortunately unlike compiled typed languages the database foreign keys are just straight up runtime overhead.
Using types correctly such as storing numbers and dates and UUID's in the actual database type rather than strings is an obvious one that typically improves space and performance, there is normally not much trade off here. Varchar with specific length I go back and forth on, it has saved me off on and catching something not validated for length properly in the app, but does get in the way when needed to expand length which happens quite often.
For coded values I like a code table per code type with foreign keys rather than say EAV, this has saved me many times in data quality and allows extra attributes to be added to specific types. It matches more closely to the type system in the client too which will have a specific type for a specific coded value. It also allows reflection like qualities where you can query the schema and follow the types. The downside is in a large app you can have many hundreds or more of code tables and its more work to cross cut features, but the benefits outweigh the downsides for me. this also goes for more full blown "entities" which are more obvious.
I don't normally use more complicated constraint or triggers, the tend to get in the way things like ETL loads and many complex rules end up working out much better in the app code. I am torn about it, but the procedural languages in the DB is usually less expressive with less reuse and can't do the other things needed such as alerting messaging etc.
Bottom line though don't be dogmatic, use the database as you can, test performance make the trade-offs. Sometimes constraints and triggers are better due to their locality to the data, sometimes leaving off foreign keys is better due to the overhead. Sometimes EAV is better for flexibility (or JSON).
In an ideal world we would run the same code in any tier(client, app server, database) for this stuff choosing the best location to run it based on the needs (or run it in multiple tiers, client for user experience, database for transactional correctness). You can almost get there with javascript an PL/V8 now days, and sort of with .Net and Sql Server, but its really not there yet.
Re: Ask HN: What's your preferred way of ensuring complex database integrity?
#15I try to keep code out of my data where possible. No triggers, no constraints (excepting maybe PRIMARY KEY and NOT NULL). Despite otherwise being a nutter for type safety, I'll even favour TEXT over enums. The main reason is change. As I evolve the code base, I might want to do something today which is principled, but seemed unprincipled yesterday. Database constraints have tended to hit me the hardest at the worst t…
Could you not then simply use a nosql solution?
Re: Ask HN: What's your preferred way of ensuring complex database integrity?
#16I am not sure that I would call these "tricks"
Re: Ask HN: What's your preferred way of ensuring complex database integrity?
#17Isn't that the primary purpose of transactions? I mean, every time you write to the database, you open a transaction, do you stuff and close it. It either works or it doesn't. If it doesn't, then it rolls back. Isn't that enough? I guess I am saying that for complex integrity checks, it should be the responsibility of the application's data layer. On the other hand what is meant by "complex"? Is it "complex" because…
Re: Ask HN: What's your preferred way of ensuring complex database integrity?
#18I try to keep code out of my data where possible. No triggers, no constraints (excepting maybe PRIMARY KEY and NOT NULL). Despite otherwise being a nutter for type safety, I'll even favour TEXT over enums. The main reason is change. As I evolve the code base, I might want to do something today which is principled, but seemed unprincipled yesterday. Database constraints have tended to hit me the hardest at the worst t…
No triggers, no constraints (excepting maybe PRIMARY KEY and NOT NULL) Could you not then simply use a nosql solution?
Re: Ask HN: What's your preferred way of ensuring complex database integrity?
#19Isn't that the primary purpose of transactions? I mean, every time you write to the database, you open a transaction, do you stuff and close it. It either works or it doesn't. If it doesn't, then it rolls back. Isn't that enough? I guess I am saying that for complex integrity checks, it should be the responsibility of the application's data layer. On the other hand what is meant by "complex"? Is it "complex" because…