Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Never write a database, even if you want to, even if you think you should

twitter.com

Post reply on HN
Print view View original

41–50 of 56 posts

Re: Never write a database, even if you want to, even if you think you should

#41

Post by btilly »

This is like the classic advice on optimization.

    The First Rule of Program Optimization: Don't do it.
    The Second Rule of Program Optimization (for experts only!): Don't do it yet.
It is because I know what a database actually does for me, and because I've had to write parts of that for myself, that I deeply understand why I don't want to write one. And why I want to push those problems to a database if I can.

Of course you should ignore my advice if you're named Jeff Dean. Look through https://research.google/people/jeff/. Among his other projects, look at BigTable, LevelDB and Spanner. All are databases, implemented for good reasons.

Re: Never write a database, even if you want to, even if you think you should

#42

Post by g-w1 »

7373737373 wrote: post #3

"Always write a database. Even if you don't want to, even if you think you shouldn't." Sometimes, naivety is a good helper when it comes to learning - too much knowledge of the task ahead can be demotivating. No one was born a professional developer either, and only by encountering and considering problems as they occur can one truly understand why certain decisions were made in the past and stuck around. Also, many…

Yes, the only way to truly learn that you should never write a database is to write a database. In fact, I bet that is what the author of the tweet did to learn that they should never write one.

I wrote a blog post about this: https://jacobgw.com/blog/observation/2023/07/08/shoot-yourse...

Re: Never write a database, even if you want to, even if you think you should

#43

Post by hfjjbf »

chomp wrote: post #10

Don’t take Twitter advice… this post does nothing but discourage play and experimentation.

The key to making breakthroughs in knowledge - either personal or societally - is to choose your battles to a degree.

There is simply too much to master, it’s a very good skill to be able to say “this is fascinating but too far afield of what I need to do”.

Alternately, choosing when to say “after dipping my toe in the water this clicks for me, I’m going to take a risk and go further” is an even harder skill to master.

This is why many people have trouble finishing a PhD dissertation. Play is a necessary step, but play without focus is a ticket to mediocrity.

Databases in particular are something one will run across in nearly any task and 99:100 times one likely shouldn’t dwell too much on what is inside the magic box.

Re: Never write a database, even if you want to, even if you think you should

#44

Post by dpritchett »

g-w1 wrote: post #42
7373737373 wrote: post #3

"Always write a database. Even if you don't want to, even if you think you shouldn't." Sometimes, naivety is a good helper when it comes to learning - too much knowledge of the task ahead can be demotivating. No one was born a professional developer either, and only by encountering and considering problems as they occur can one truly understand why certain decisions were made in the past and stuck around. Also, many…

Yes, the only way to truly learn that you should never write a database is to write a database. In fact, I bet that is what the author of the tweet did to learn that they should never write one. I wrote a blog post about this: https://jacobgw.com/blog/observation/2023/07/08/shoot-yourse...

Good ear. She is cofounder of a company that built their own “distributed column store”: https://youtu.be/tr2KcekX2kk?si=ID0qB-O2ucXF4GXC

Re: Never write a database, even if you want to, even if you think you should

#45

Post by benj111 »

derefr wrote: post #36
rewmie wrote: post #20

Earlier quoted context omitted.

> But know that making a real production database tends to be harder than it looks, and so tends to be all-consuming, and a team that depends on a custom database tends to become a database team rather than an anything else team. The original statement does not include any caveat or constraint, and is an absolute statement that does not depend on specifics. The statement is not "Never write a custom database as a sub…

It is a statement directed at a readership of — and therefore assuming it will only be read by — application developers, rather than systems programmers; and so the context of "Never write a custom database as a subproject of any other project" is implicit. Also, you're reacting to an incomplete quote. The tweet is: > "Never write a database. Even if you want to, even if you think you should. Resist. Never write a da…

>Unless you have to write a database. But you don't.

The quote you quoted says you don't need to write a database, so if you want to agree, you also have to agree that Amazon and Google didn't need to write databases.

And learning would seem to be covered by the "want to" or "should"

Re: Never write a database, even if you want to, even if you think you should

#46

Post by derefr »

benj111 wrote: post #45
derefr wrote: post #36

Earlier quoted context omitted.

It is a statement directed at a readership of — and therefore assuming it will only be read by — application developers, rather than systems programmers; and so the context of "Never write a custom database as a subproject of any other project" is implicit. Also, you're reacting to an incomplete quote. The tweet is: > "Never write a database. Even if you want to, even if you think you should. Resist. Never write a da…

>Unless you have to write a database. But you don't. The quote you quoted says you don't need to write a database, so if you want to agree, you also have to agree that Amazon and Google didn't need to write databases. And learning would seem to be covered by the "want to" or "should"

Again, pedantry: Amazon and Google aren't people; they aren't reading advice on the Internet. So the "you" in this statement will never apply to them. It only applies to individuals reading the statement. And, indeed, "you" do not need to write a database. Some company might very well need to write a database; and therefore coerce one or more of its employees into writing a database. But, like the statement says, the employees should resist. Very likely the company is wrong about needing to write a database, and resistance will help it discover that.

However, every once in a while, even after everyone resists for months/years, it will turn out that the company really does need to write a database, and so — still mostly against their will — the individuals employed there will set out to write a database. (And once they've "written a database", and it turned out to actually be the best course of action for the company, then the advice no longer applies — because they're no longer considering "writing a database" at that point, but rather maintaining an existing database, that they happened to be the creators of.)

> And learning would seem to be covered by the "want to" or "should"

Instrumental vs. terminal goals. If I want to do X, then I need to do Y to achieve X. If I want to learn how databases work, then I need to write a database, because there's no other good way.

Re: Never write a database, even if you want to, even if you think you should

#47

Post by myshpa »

timeon wrote: post #40
ncphillips wrote: post #28

Earlier quoted context omitted.

The tweet is part of a conversation about building a database for your business, it has nothing to do with play and experimentation for the sake of learning

Link is not working for me. Title uses word `never`.

Michał Niczyporuk @mihn

So when talk "Never write a database" is coming? ;)

Charity Majors @mipsytipsy

That's it. That's the talk.

"Never write a database. Even if you want to, even if you think you should. Resist. Never write a database. Unless you have to write a database. But you don't."

I will present this talk at any conference of your choosing.

Re: Never write a database, even if you want to, even if you think you should

#48

Post by filleokus »

I don't know the history / timeline of Honeycomb, but was FoundationDB available open source when they decided to write their own DB?

If I ever would find myself walking into building-my-own-database-territory, I would probably first try to reach for FoundationDB and see if that could work.

I've just played around with it, but it seems like a great tool for building high performant and robust distributed databases on top of.

Re: Never write a database, even if you want to, even if you think you should

#49

Post by slowmovintarget »

btilly wrote: post #41

This is like the classic advice on optimization. The First Rule of Program Optimization: Don't do it. The Second Rule of Program Optimization (for experts only!): Don't do it yet. It is because I know what a database actually does for me, and because I've had to write parts of that for myself, that I deeply understand why I don't want to write one. And why I want to push those problems to a database if I can. Of cour…

Or if you're Rich Hickey (Datomic).

Re: Never write a database, even if you want to, even if you think you should

#50

Post by benj111 »

derefr wrote: post #46
benj111 wrote: post #45

Earlier quoted context omitted.

>Unless you have to write a database. But you don't. The quote you quoted says you don't need to write a database, so if you want to agree, you also have to agree that Amazon and Google didn't need to write databases. And learning would seem to be covered by the "want to" or "should"

Again, pedantry: Amazon and Google aren't people; they aren't reading advice on the Internet. So the "you" in this statement will never apply to them. It only applies to individuals reading the statement. And, indeed, "you" do not need to write a database. Some company might very well need to write a database; and therefore coerce one or more of its employees into writing a database. But, like the statement says, the…

>If I want to do X, then I need to do Y to achieve X.

And you accuse me of pedantry? This then comes under need then?!?

>Amazon and Google aren't people

No, but the people within them make decisions, ie decisions to write a database. So the 'you' would be directed at the person in the organisation who decided to write the database.

Post reply on HN