Live data from Hacker News

Ask HN: Do you use foreign keys in relational databases?

news.ycombinator.com

201–210 of 251 posts

Re: Ask HN: Do you use foreign keys in relational databases?

#201

No, especially on large tables with billions of records. They make online schema changes impossible. More details: https://github.com/github/gh-ost/issues/331#issuecomment-266...

Curious why no one else had mentioned this. Are they using online schema transforms besides gh-ost and Percona? Does Postgres handle this better than MySQL?

Postgres has index concurrently which is better or few other instant things which MySQL 8 also has but you still need to be careful.

But some things like adding a column with default value is still hard.

https://www.braintreepayments.com/blog/safe-operations-for-h... this article has a good guideline on this.

Re: Ask HN: Do you use foreign keys in relational databases?

#203

Your database is is the state of your system. Guard it! I just ran into severe data corruption at a large client because a programmer four years ago wrote an empty catch block. The system would open a transaction, hit a fault, roll back, then continue writing to the database as if it’s still in the context of the transaction. I spent some time trying to pin down exactly what it did, and found that many writes went th…

You're exactly right. This is the crux of it. In many domains (not every domain, but every one I've ever worked in) you can delete/rewrite/change the app code without much fuss but the data is vital.

For that reason having constraints enforced by the system that stores the data, external to the app code which developers will inevitably mess up, is so useful and important.

So many issues in my experience have been similar to the one you describe. If the right constraints had been present so much work and so many headaches could have been avoided.

I feel like people who claim they're not needed or not important or "the app code will do it" need to be wrangled into maintenance work of old systems for a year or two until they repent. Rather than boshing out an ill-considered prototype and then moving on before the bugs are discovered.

Re: Ask HN: Do you use foreign keys in relational databases?

#204
For me it's a casevof pick your poison.

If you have no FK, the day will come that your data corrupts. Tiny application bug, wrong manual data fix, incomplete datamodel communication. It's a case of when, not if. And when it happens, it can fester for weeks or months, corrupting all kinds of data in unfixable ways.

If you have FK, you can go all-in on relational. The stuff is so powerfull, you'd be dumb not to. But then comes the too-smart-optimizer problem. Just like with compilers and opengl shaders, the tiniest change might make you fall off the optimizer's preferred path and get a way-to-slow version. Even when no code has changed, a minor DB version or even a tiny shift in statistics will kill you.

Personally, I tend to consider integrity more important than performance, but both have a zone where they are good enough to make trading off worthwile.

So I do FKs and joins, and count on rigorous testing and monitoring to keep things in check. Someone else might decide otherwise. Both strategies require you to do the unsexy part of the wirk, or a harsh punishment will follow.

Re: Ask HN: Do you use foreign keys in relational databases?

#205
post #16

Earlier quoted context omitted.

Isn't that just storing the value in a normal column?

What I'm specifically looking for is a foreign key relation so that you could for example still generate a chart of relations. For example an audit log you wouldn't want the consistency check, but it would still be nice to know it links to X tables.

In the RDBMSes I'm aware of, you can disable foreign key checks while still adding the constraints themselves---they won't do anything as far as the DBMS is concerned, but can presumably still be picked up by whatever tool you are using to generate this link information.

Re: Ask HN: Do you use foreign keys in relational databases?

#206
post #153

> He rather have a smooth data migration process than having an unexpected error and abort the whole operation that the migration is only a small part of. that's some take, because if that "unexpected error" is not reported because FK integrity was turned off, that means the data containing integrity errors goes right in. Now your database is corrupt. Dealing with a DB where random rows here and there are not conform…

This is one of the many problems with "senior" engineers who have been in the industry four years over three different jobs - they don't stay long enough to see the consequences of their shitty code.

Re: Ask HN: Do you use foreign keys in relational databases?

#207
> His main argument is difficulties during data migrations which he frequently encounters. He rather have a smooth data migration process than having an unexpected error and abort the whole operation that the migration is only a small part of.

NUTS!

> I suspect the errors might be mainly caused by not considering data integrity at all at the first place

Spot on.

----

I deal with interface with MANY ERP-like software with as many "database designs" as you can't imagine, and have face the fun of interface with many system in my long career.

Among DOZENS only 2 major packages have zero issues and are the only ones with competent schema designs.

Let me tell you how bad is it:

One of my interfaces is with a package that, somewhat, manage to mix all the ways to be wrong about RDBMS:

- MySql, not a good start

- Only Strings and Ints datatypes, and the Ints datatypes are not many

- This means ALL data is suspected (dates, money, and yes, strings too)

- The tables are named `UNCC_00001, UNCC_00002...`

- The fields are named `FT_0001, FT_0002...` and obviously the first is the PK. This pk is purely advisory, other fields can be part of the PK but you can't know looking at the schema

- NO FK at all, so ALL TABLES have integrity issues.

- NO indexes, because why make joins or lookups faster?

- Weird normalization, like yes, no, maybe?

- And no option to let me add some sanity to the DB, because the database is (in my niches) "property" of the software makers so can't be touched, only queried.

And not tell the rest, because this is only the query side, the CRUD side is nuts, and not wanna remember much about it.

---

RDBMS is one of the most simplest ways, to get FOR FREE, and minimal effort, a sane, well behaved and performant system.

Refuse to use them is NUTS. Refuse to use the abilities they have is NUTS.

I only have ONE actual company (in +20 years) with a real reason to go NoSQL, and that only was for a fraction of the thing (in fact, the data was mirrored to a RDBMS for reporting!). And maybe that was true 5-10 years ago, today I think modern SQL engines catchup more and more on the scenarios (Today I have used timescale just for speed up log processing, and was so much nicer that deal with weird log storage engines).

Re: Ask HN: Do you use foreign keys in relational databases?

#208

Earlier quoted context omitted.

We are absolutely talking about billions of records. But when you say "return excess data to the application" I'm not sure what you mean. Not doing lots of complex JOINs doesn't mean not filtering the queries at the DB at all. Nobody is pulling back a billion records at a time. Here's an example of what I'm talking about - read the first comment on https://www.brentozar.com/archive/2015/05/do-foreign-keys-ma... , ano…

> We are absolutely talking about billions of records. > But when you say "return excess data to the application" I'm not sure what you mean. Not doing lots of complex JOINs doesn't mean not filtering the queries at the DB at all. Nobody is pulling back a billion records at a time. The question then becomes "Are you gaining anything by not using foreign keys on the database?" What is the additional speed impact of JO…

I'll cop to my situation being unusual, but that's what I said in response to other posts in this thread: so is everyone's. This isn't a topic with a blanket rule. I loaded my original post with caveats like "for me" to try to make that clear. Sorry if it wasn't.

Apart from your skepticism about whether my org's DB is as efficient as could be, I don't think we actually disagree, unless your argument is that people in my situation are somehow obligated to make a specific set of choices rather than what works for them.

Re: Ask HN: Do you use foreign keys in relational databases?

#209
post #143

I agree with your colleague, and I insist on pushing my car everywhere because I fear gas as it is flammable. In other words, the world is full of idiots; and any time I start forgetting about it, I read something like your post and I get a wake-up call. What does R stand for in RDBMS is you don't use foreign keys and joins? Please, keep using your FKs, stay safe and don't mingle too much with idiots.

I think the author is talking about 'foreign key constraints' - You could have foreign keys without enforcing a constraint. Personally, I don't use foreign key constraints because: 1. It makes schema migrations and other data-management operations more difficult. 2. On insertion, the database needs to perform an additional check to verify that the record exists at the foreign key; this carries a performance cost; IMO…

On point 3 it should be noted that it's almost always a mistake to optimize for scale at the start of a projects lifetime. There will be exceptions, but in general this is true.

You can always migrate that data to a more useful format if you find it starts hurting you at scale, if you start with the assumption you need the scale you're hurting yourself in the here and now for theoretical future benefit.

> The real reason people use joins is because they want to pack a lot of details onto the user's screen when they are looking at a list view

This is completely, emphatically wrong. I'm somewhat miffed at the air of authority you're using here. People use joins for the normalization of data.

Re: Ask HN: Do you use foreign keys in relational databases?

#210
post #179

Earlier quoted context omitted.

> I prefer to assemble data on the front end as much as possible because it allows my REST API calls to be granular It's clear you have never work with a lot of data. > The real reason people use joins is because they want to pack a lot of details onto the user's screen I hate this illusion that web programming is the whole of software development.

> It's clear you have never work with a lot of data. Sure, I only wrote an open source distributed pub/sub system with channel-based sharding which has been used by thousands of companies to support hundreds of thousands of concurrent users, but I guess 'lots of data' is a relative term.

That has nothing to do with data or data modeling.
Post reply on HN