I've always wanted to write a database
Do maps count?
Never write a database, even if you want to, even if you think you should
11–20 of 56 posts
Re: Never write a database, even if you want to, even if you think you should
#12what does "writing a database" mean?
Building/developing/creating a https://en.wikipedia.org/wiki/Database [management system].
of course. but i would differentiate between "writing a new database engine" and "designing a database to be implemented using an existing engine (e.g. oracle)"
Re: Never write a database, even if you want to, even if you think you should
#13"Always write a database. Even if you don't want to, even if you think you shouldn't." Sometimes, naivety is a good helper when it comes to learning - too much knowledge of the task ahead can be demotivating. No one was born a professional developer either, and only by encountering and considering problems as they occur can one truly understand why certain decisions were made in the past and stuck around. Also, many…
I don't think most people would disagree with that. If you want to build a database, for curiosity, or to solve a particular problem, or just because it seems fun, go ahead. But know that making a real production database tends to be harder than it looks, and so tends to be all-consuming, and a team that depends on a custom database tends to become a database team rather than an anything else team. That could be the right thing - after all building databases as an explicit goal has lead to a lot of very good businesses - but you need to go into it with your eyes open. Building a production-quality database is a long road.
The same is true of some other areas, like durable storage systems, filesystems, and compute isolation/virtualization systems. In all these cases going from zero to "undergrad quality demo" is doable in a couple days. Going from demo to running something in production is vastly harder and more expensive and requires a completely different mind set. Building the "undergrad quality demo" is a great exercise I'd recommend to anybody curious about these areas. The road to production, however, is a tougher one.
Source: I build and maintain databases for a living at AWS. One of my previous side projects (https://www.usenix.org/conference/nsdi20/presentation/brooke...) turned out to be bigger than expected and has a whole team dedicated to it. That one (Physalia) is even quite far from being a general database (for example it doesn't support SQL or cross-shard transactions).
Re: Never write a database, even if you want to, even if you think you should
#14Earlier quoted context omitted.
Building/developing/creating a https://en.wikipedia.org/wiki/Database [management system].
of course. but i would differentiate between "writing a new database engine" and "designing a database to be implemented using an existing engine (e.g. oracle)"
I updated the reply.
Re: Never write a database, even if you want to, even if you think you should
#15Of course, in production, it is really hard to find a use case that isn’t served ‘well enough’ by existing database technology.
But: almost ever should you say never.
What better way to learn indexing tradeoffs than to write your own database?
What existed before Redis?
What existed before LevelDB?
What existed before Git?
Sure, write a database if you want. But be humble about the effort involved and reliability you’ll get.
Re: Never write a database, even if you want to, even if you think you should
#16Back in the 1990s I worked at a financial services company, all our back-office code used key/value (index files) storage. I don't remember ever doing any kind of explicit record locking for updates, it was always just "last write wins" and we never seemed to have much of an issue with it. It was early in my career so there was a big component of not knowing what I didn't know at that time.
Re: Never write a database, even if you want to, even if you think you should
#17"Always write a database. Even if you don't want to, even if you think you shouldn't." Sometimes, naivety is a good helper when it comes to learning - too much knowledge of the task ahead can be demotivating. No one was born a professional developer either, and only by encountering and considering problems as they occur can one truly understand why certain decisions were made in the past and stuck around. Also, many…
I don't think most people would disagree with that. If you want to build a database, for curiosity, or to solve a particular problem, or just because it seems fun, go ahead. But know that making a real production database tends to be harder than it looks, and so tends to be all-consuming, and a team that depends on a custom database tends to become a database team rather than an anything else team. That could be the…
[dead]
Re: Never write a database, even if you want to, even if you think you should
#18Every database you've ever benefited from was started by someone who didn't heed this advice!
That said, it's a hard job to do right and there are now databases of every different size made by people who are just much better than me.
Re: Never write a database, even if you want to, even if you think you should
#19It's like the saying about game engines. You should either set out to develop a database or you should develop an app that needs a database. If you try and do both at once you will never finish or you will end up with a half baked database.
Re: Never write a database, even if you want to, even if you think you should
#20"Always write a database. Even if you don't want to, even if you think you shouldn't." Sometimes, naivety is a good helper when it comes to learning - too much knowledge of the task ahead can be demotivating. No one was born a professional developer either, and only by encountering and considering problems as they occur can one truly understand why certain decisions were made in the past and stuck around. Also, many…
I don't think most people would disagree with that. If you want to build a database, for curiosity, or to solve a particular problem, or just because it seems fun, go ahead. But know that making a real production database tends to be harder than it looks, and so tends to be all-consuming, and a team that depends on a custom database tends to become a database team rather than an anything else team. That could be the…
> But know that making a real production database tends to be harder than it looks, and so tends to be all-consuming, and a team that depends on a custom database tends to become a database team rather than an anything else team.
The original statement does not include any caveat or constraint, and is an absolute statement that does not depend on specifics. The statement is not "Never write a custom database as a subproject of any other project". The original statement is "Never write a database." Like in any conceivable scenario, including a project whose main goal is to write a database.
I also feel that this sort of clickbait advice contrasts heavily with the history of SQLite.