The Xeet seemed to be against writing an engine, not an application.
Are we not really integrators of prior art, for the most part?
Never write a database, even if you want to, even if you think you should
51–56 of 56 posts
Re: Never write a database, even if you want to, even if you think you should
#52Earlier quoted context omitted.
Again, pedantry: Amazon and Google aren't people; they aren't reading advice on the Internet. So the "you" in this statement will never apply to them. It only applies to individuals reading the statement. And, indeed, "you" do not need to write a database. Some company might very well need to write a database; and therefore coerce one or more of its employees into writing a database. But, like the statement says, the…
>If I want to do X, then I need to do Y to achieve X. And you accuse me of pedantry? This then comes under need then?!? >Amazon and Google aren't people No, but the people within them make decisions, ie decisions to write a database. So the 'you' would be directed at the person in the organisation who decided to write the database.
> And you accuse me of pedantry?
...no? I was saying that my own response is pedantry, rather than being an argument that tries to address the spirit of your argument.
> This then comes under need then?!?
Yes. The definition of "need" is "something that some higher-level goal cannot be achieved without." Ultimately, at the terminal level, everything is a preference; you want to continue living, companies want to not go bankrupt, etc. Everything required to achieve those terminal preferences, are needs. You need to eat if you want to live. A company needs to make money if it wants to avoid bankruptcy. Etc.
> No, but the people within them make decisions, ie decisions to write a database. So the 'you' would be directed at the person in the organisation who decided to write the database.
In a bigcorp, ultimately, a manager or business analyst decided that someone else should write them a database. Nobody decided to, themselves, write a database.
Re: Never write a database, even if you want to, even if you think you should
#53It is remarkable how approximately 50% of the things Charity Majors says resonate so well with me … and how the other 50% irritate the ever-loving far out of me.
I think what she means here is “nobody ELSE should write a database,” because they certainly found it necessary and beneficial to do so.
(In all seriousness, usually you shouldn’t write a database. But somebody has to, you know? There’s no need to gate-keep like that.)
Re: Never write a database, even if you want to, even if you think you should
#54[flagged]
Re: Never write a database, even if you want to, even if you think you should
#55I don't know the history / timeline of Honeycomb, but was FoundationDB available open source when they decided to write their own DB? If I ever would find myself walking into building-my-own-database-territory, I would probably first try to reach for FoundationDB and see if that could work. I've just played around with it, but it seems like a great tool for building high performant and robust distributed databases on…
According to GitHub, no, but it may have been available before then. Regardless, the design of Honeycomb's database(store?) aligns more with an analytical workload, which FoundationDB lists as a non-goal. It may not have been a good fit then or even now.
Re: Never write a database, even if you want to, even if you think you should
#56Of course, in production, it is really hard to find a use case that isn’t served ‘well enough’ by existing database technology. But: almost ever should you say never. What better way to learn indexing tradeoffs than to write your own database? What existed before Redis? What existed before LevelDB? What existed before Git? Sure, write a database if you want. But be humble about the effort involved and reliability you…
BerkeleyDB existed before Redis, LevelDB, and Git and was widely used as a database. SVN originally used BDB as its storage layer (but switched when people complained it was too hard to build BDB). https://en.wikipedia.org/wiki/Berkeley_DB contains people-decades of hard-won experience on how to build a simple database system. I would consider SQLite, itself 20+ years old now, the intellectual heir of BDB. It was as…
Yes, thanks: the history is fun and instructive.
My questions were rhetorical, BTW: emphasizing that progress happens when what exists now doesn't stop someone from making something different and maybe better.
We're probably not disagreeing. To summarize:
- practically, given real-world constraints and risk tolerances, yes, very few people should write databases for production. (Except villains. They should expend tremendous energy and effort writing bespoke artisanal databases, resulting in less time to be evil.)
- educationally, there is a lot to be gained by writing your own, even if only a small part [1], such as something around indexing, a write-ahead-log, a query planner, or even a query parser [2]
- if everyone thought e.g. 'writing a new database isn't worth it', collectively we'd be in a worse place
FWIW, I have a soft place in my heart for all the great projects that come out of Berkeley.
Note 1: Let me propose a game for a group of people. One person names some narrow aspect of a database, perhaps in the hopes that it is boring, trivial, or "solved". (Perhaps "autoincrementing indexes" for example.) Then the other people to talk about all the ways that the thing is actually quite hard, interesting, non-obvious, and maybe (?) even a good research area.
Note 2: It is nice that SQL is declarative. AND there is room for improvement, starting with composability. See https://news.ycombinator.com/item?id=24730713