Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL is enough

gist.github.com

Post reply on HN
Print view View original

111–120 of 323 posts

Re: PostgreSQL is enough

#111

Post by niels_bom »

legohead wrote: post #78
superb-owl wrote: post #2

I often go down rabbit holes like this, trying to collapse and simplify the application stack. But inevitably, as an application grows in complexity, you start to realize _why_ there's a stack, rather than just a single technology to rule them all. Trying to cram everything into Postgres (or lambdas, or S3, or firebase, or whatever other tech you're trying to consolidate on) starts to get really uncomfortable. That s…

My saying has always been: be nice to the DB Don't use it anymore than you have to for your application. Other than network IO it's the slowest part of your stack.

Would you say it's slower than file IO too?

Re: PostgreSQL is enough

#112

Post by jakey_bakey »

bun_terminator wrote: post #97
acaloiar wrote: post #93

Earlier quoted context omitted.

It's a decent question. I believe there's a simple answer that explains a large part of the reason people choose databases over flat files. People want ACID compliance _and_ data distribution, and databases are traditionally the only things that provide both.

I have never heard the term ACID so I don't believe that's a reason for us at least

tl;dr it's an acronym that explains how the operations happening inside a database are reliable.

For example, when making a bank transfer, the database guarantees that the update doesn't happen twice by accident.

Re: PostgreSQL is enough

#113

Post by bun_terminator »

afandian wrote: post #109
bun_terminator wrote: post #95

Earlier quoted context omitted.

ok I get that, but at least our applications usually only have one thing (one thread of one process on one machine) operating on the database. So maybe it's just our silly usecase

It's a narrow use-case that doesn't cover RDMS. SQLite fits it though.

The first thing I did was ask why we don't use sqlite or at least postgres. The answer was that they are free and therefore we don't trust them. So Oracle it is. Which is bananas because our customers have to buy an oracle license, which is money that we don't get. Pretty wild

Re: PostgreSQL is enough

#114

Post by throwawaaarrgh »

If you twist yourself into a pretzel enough you can make Bash do all these things. Doesn't make it a good idea.

The biggest problem with depending on Postgres is it's a large complex monolith. Any problem you have only has two possible solutions: 1) spend a ton of time trying to twist yourself into a more pretzely shape to get it to do what you want, or 2) replace that thing you wanted with some external thing.

Both of these waste valuable time on something that isn't providing any business value. They also are entirely preventable/avoidable, by simply not putting all your eggs into one basket.

Basically the whole "just use Postgres" philosophy boils down to: I don't want to learn new things, and I just like making things from scratch with custom code. That's great if you're an engineer and you want job security. But it's bad strategy, engineering, and use of time and resources. Anyone who approves of using Postgres in this way should not be in charge of engineering decisions.

Re: PostgreSQL is enough

#115

Post by emseetech »

randomdata wrote: post #105
tangjurine wrote: post #100

Earlier quoted context omitted.

> Postgres suffers from the n+1 problem, while SQLite does not. ?

Indeed.

It’s worth elaborating:

N+1 Queries Are Not A Problem With SQLite

https://www.sqlite.org/np1queryprob.html#:~:text=N%2B1%20Que....

Re: PostgreSQL is enough

#116

Post by marcosdumay »

bun_terminator wrote: post #95
treflop wrote: post #91

Earlier quoted context omitted.

Well a database is just a binary file format with an API attached on. Because having hundreds or thousands of concurrent file handles across a data center is kind of hard.

ok I get that, but at least our applications usually only have one thing (one thread of one process on one machine) operating on the database. So maybe it's just our silly usecase

That's unusual. The rule is that you replicate the database consumers lots and lots of times.

Still then, a database will improve your data access, validate your data, and provide interop in case you need to use the file with different software.

It's also a very high quality serialization library, that replaces one of the most vulnerability-enabling layers of your software. (But then, I've just read you use Oracle, so maybe forget that one.)

Re: PostgreSQL is enough

#117

Post by crabbone »

bun_terminator wrote: post #87

As a hardcore c++ guy, I recently switched to a company heavily into databases. I never had contact to databases before. And I'd like to go one step further: Why databases? I come from an industry that heavily uses custom binary file formats. And I'm still bewildered by the world of databases. They seem to solve many issues on the surface, but not really in pratice. The heavy limitations on data types, the update dis…

Over years, I've seen a lot of poorly made binary formats used by various programs. Databases like PostgreSQL are where such binary formats had evolved through benchmarking, testing for various edge cases, and are generally better than whatever you will find in the bespoke formats developed for a particular program.

You have to be really good at understanding how storage works, have a lot of time and resources to develop your own program to beat something like PostgreSQL. They have zillions of man-hours on you when you start, experience and knowledge. It's not impossible that you could find a case where a bespoke format would beat an established storage product, but over a range of cases, you most likely won't.

And, of course, there's a convenience aspect. Outside of niche technologies, SQL offers the richest language for data description and operations on data.

As for the inconsistencies between SQL implementations: in practice, it matters very little: most programs will never migrate between different SQL implementations anyways.

As for upgrades: it's a doubly-edged sword. You get a very expressive data format and it's not surprising that it's hard to upgrade. But, try to match the abilities of SQL in your own format, and you'll probably find out that it's hard to have generic tools for upgrading it too.

None of this means that you cannot do better than SQL databases. It'd be ridiculous to think there could be a way to prove that SQL is somehow the best we can get. It's just that it's very hard to do better. Especially if you want a universal tool

Re: PostgreSQL is enough

#118

Post by bsdpufferfish »

bun_terminator wrote: post #87

As a hardcore c++ guy, I recently switched to a company heavily into databases. I never had contact to databases before. And I'd like to go one step further: Why databases? I come from an industry that heavily uses custom binary file formats. And I'm still bewildered by the world of databases. They seem to solve many issues on the surface, but not really in pratice. The heavy limitations on data types, the update dis…

Read the original relational database paper by Codd. The problem is that trees of data are not flexible for querying, you really need a graph. If you draw it out on paper, you will find a relation is a very efficient way to store general graphs. I'm actually suprised to hear this from a C++ engineer, because sorting and searching columns of data is the name of the game in C++.

Another key problem they solve is separating the logical representation from the on disk representation. You can evolve and and optimize storage without breaking anything.

The other problem with files is that you have a disconnect between in memory and on disk. You have to constantly serialize and deserialize. Sqlite has quite a bit of info on this: https://www.sqlite.org/appfileformat.html

Re: PostgreSQL is enough

#119

Post by runningamok »

Ensorceled wrote: post #39
emseetech wrote: post #22

This makes a strong case, but I've decided to start every new project with sqlite and not switch until absolutely necessary. If Postgres is the 90% case, then sqlite is the 80% case and is also dead simple to get going and genuinely performant. So when vertical scaling finally fails me, I know I'll be at a wonderful place with what I'm building.

> [...] sqlite is the 80% case and is also dead simple to get going and genuinely performant. I don't understand this. PostgreSQL is ALSO dead simple to get going, either locally or in production. Why not just start off at 90%? I mean, I get there are a lot of use cases where sqlite is the better choice (and I've used sqlite multiple times over the years, including in my most recent gig), but why in general?

One use case where SQLite is a good option is for embedding as a local database in an app. Starting local-only with SQLite allows you to defer a lot of the backend effort while testing an MVP.

Re: PostgreSQL is enough

#120

Post by Kinrany »

benrutter wrote: post #8

On the [Simplify: move code into databases]( https://sive.rs/pg ), has anyone actually tried this? My instinct is that the output won't be simpler at all, but a big twisty web of varying parts and competing use cases, all extremely hard to discover. To me, having two things that do something distinct is simpler, and ironically, Rich Hickey says this exact thing in Simplicity Matters, which is quoted in the article.

Having two distinct things doesn't mean placing them in separate processes. Or we'd all be writing Erlang by now
Post reply on HN