Live data from Hacker News

Do you even need a database?

dbpro.app

31–40 of 311 posts

Re: Do you even need a database?

#31
Separate from performance, I feel like databases are a sub-specialty that has its own cognitive load.

I can use databases just fine, but will never be able to make wise decisions about table layouts, ORMs, migrations, backups, scaling.

I don't understand the culture of "oh we need to use this tool because that's what professionals use" when the team doesn't have the knowledge or discipline to do it right and the scale doesn't justify the complexity.

Re: Do you even need a database?

#32
post #21

Earlier quoted context omitted.

Only if you get there and need it.

but it's so trivial to implement SQLite, in almost any app or language...there are sufficient ORMs to do the joins if you don't like working with SQL directly...the B-trees are built in and you don't need to reason about binary search, and your app doesn't have 300% test coverage with fuzzing like SQLite does you should be squashing bugs related to your business logic, not core data storage. Local data storage on you…

Came here to also throw in a vote for it being so much easier to just use SQLite. You get so much for so very little. There might be a one-time up-front learning effort for tweaking settings, but that is a lot less effort than what you're going to spend on fiddling with stupid issues with data files all day, every day, for the rest of the life of your project.

Re: Do you even need a database?

#35
post #23

I avoided DBs like the plague early in my career, in favor of serialized formats on disk. I still think there's a lot of merit to that, but at this point in my career I see a lot more use case for sqlite and the relational features it comes with. At the least, I've spent a lot less time chasing down data corruption bugs since changing philosophy. Now that said, if there's value to the "database" being human readable/…

> Dealing with even sqlite is a pain in the ass when you just need to tweak or read something, especially if you're not the dev. How? With SQL is super easy to search, compare, and update data. That's what it’s built for.

Pain in the ass was way too strong, I retract that. Mainly I meant relative. For example `nvim .json` and then /search for what I want, versus tracking down the sqlite file, opening, examining the schema, figuring out where the most likely place is that I care about, writing a SQL statement to query, etc.

Re: Do you even need a database?

#36
I've just built myself a useful tool which now really would benefit from a database and I'm deeply regretting not doing that from the get-go.

So my opinion has thoroughly shifted to "start with a database, and if you _really_ don't need one it'll be obvious.

But you probably do.

Re: Do you even need a database?

#37
post #21

Earlier quoted context omitted.

Only if you get there and need it.

but it's so trivial to implement SQLite, in almost any app or language...there are sufficient ORMs to do the joins if you don't like working with SQL directly...the B-trees are built in and you don't need to reason about binary search, and your app doesn't have 300% test coverage with fuzzing like SQLite does you should be squashing bugs related to your business logic, not core data storage. Local data storage on you…

Honestly, there is zero chance you will implement anything close to sqlite.

What is more likely, if you are making good decisions, is that you'll reach a point where the simple approach will fail to meet your needs. If you use the same attitude again and choose the simplest solution based on your _need_, you'll have concrete knowledge and constraints that you can redesign for.

Re: Do you even need a database?

#38

I agree. Databases are useless. You don't even need to load it into the memory. Reading it from the disk when there is a need to read something must be ok. I don't believe the case that there are billions of records so the database must be something optimized for handling it. That amount of records most likely is something like access logs etc, I think they should not be stored at all, for such case. Even it's postgr…

I worked as a software engineer for 30 years before being forced to use a database, and that was for a web site. I've been coding actively, daily, since the 70's. Forever we just wrote proprietary files to disk, and that was the norm, for decades. Many a new developer can't even imagine writing their own proprietary file formats, the idea literally scares them. The engineers produced today are a shadow of what they u…

Yeah, it scares me because I'm experienced enough to know all the difficulties involved in keeping durable data consistent, correct, and performant

Re: Do you even need a database?

#40

Earlier quoted context omitted.

Only if you get there and need it.

Exactly. And most apps don't get there and therefore don't need it.

Your article completely ignores operational considerations: backups, schema changes, replication/HA. As well as security, i.e. your application has full permissions to completely destroy your data file.

Regardless of whether most apps have enough requests per second to "need" a database for performance reasons, these are extremely important topics for any app used by a real business.

Post reply on HN