Live data from Hacker News

PostgreSQL is enough

gist.github.com

101–110 of 323 posts

Re: PostgreSQL is enough

#101

This makes a strong case, but I've decided to start every new project with sqlite and not switch until absolutely necessary. If Postgres is the 90% case, then sqlite is the 80% case and is also dead simple to get going and genuinely performant. So when vertical scaling finally fails me, I know I'll be at a wonderful place with what I'm building.

I’m with you in general, but what about vector search? It really feels like the DB industry has taken a huge step backward from the promise of SQL. Switching from Postgres to SQLite is easy because the underlying queries are at least similar. But as soon as you introduce embeddings, every system is totally different (and often changing rapidly).

Just use SQLite?

Specialized vector indexes become important when you have a large number of vectors, but the reality of software is that it is unlikely that your application will ever be used at all, let alone reach a scale where you start to hurt. Computers are really fast. You can go a long way with not-perfectly-optimized solutions.

Once you have proven that users actually want to use your product and see growth on the horizon to where optimization becomes necessary, then you can swap in a dedicated vector solution as needed, which may include using a vector plugin for SQLite. The vector databases you want to use may or may not use SQL, but the APIs are never that much different. Instead of one line of SQL to support a different implementation you might have to update 5 lines of code to use their API, but we're not exactly climbing mountains here.

Know your problem inside and out before making any technical choices, of course.

Re: PostgreSQL is enough

#102

Earlier quoted context omitted.

Postgres complicates the application in several ways. In particular, Postgres suffers from the n+1 problem, while SQLite does not. That requires a significant amount of added complexity in the application to hack around. Why over engineer the application before it has proven itself as something anyone even wants to use? Let's face it, the large majority of software written gets thrown away soon after it is created. I…

> Postgres suffers from the n+1 problem, while SQLite does not. ?

[deleted]

Re: PostgreSQL is enough

#103

This makes a strong case, but I've decided to start every new project with sqlite and not switch until absolutely necessary. If Postgres is the 90% case, then sqlite is the 80% case and is also dead simple to get going and genuinely performant. So when vertical scaling finally fails me, I know I'll be at a wonderful place with what I'm building.

I’m with you in general, but what about vector search? It really feels like the DB industry has taken a huge step backward from the promise of SQL. Switching from Postgres to SQLite is easy because the underlying queries are at least similar. But as soon as you introduce embeddings, every system is totally different (and often changing rapidly).

[deleted]

Re: PostgreSQL is enough

#104
Specifically with PostgreSQL, number of connections is still the killer (if you don’t have smart proxies). So you can’t be cavalier about putting as many use cases as possible.

For example, when using PG pub/sub, you will run out of connections quick.

Generally, all DBMS needs a smart self adjusting query killer. Without it, one bad query will ruin it for everyone.

Re: PostgreSQL is enough

#105

Earlier quoted context omitted.

Postgres complicates the application in several ways. In particular, Postgres suffers from the n+1 problem, while SQLite does not. That requires a significant amount of added complexity in the application to hack around. Why over engineer the application before it has proven itself as something anyone even wants to use? Let's face it, the large majority of software written gets thrown away soon after it is created. I…

> Postgres suffers from the n+1 problem, while SQLite does not. ?

Indeed.

Re: PostgreSQL is enough

#106

As a hardcore c++ guy, I recently switched to a company heavily into databases. I never had contact to databases before. And I'd like to go one step further: Why databases? I come from an industry that heavily uses custom binary file formats. And I'm still bewildered by the world of databases. They seem to solve many issues on the surface, but not really in pratice. The heavy limitations on data types, the update dis…

> They seem to solve many issues on the surface, but not really in practice

I believe you haven't had a chance to work on problems that require an actual database. Multi-user access, ACID support, unified API (odbc/jdbc), common query language... all of these would require many man-years to set properly with a custom solution.

> the update disasters, the incompatibility between different SQL engines etc all make it seem like an awful idea

What update disasters? If you meant by updating database versions, these aren't things you do frequently because the database is expected to be running 24/7. But Postgres and Mysql are already rock solid here. Wrt SQL engine incompatibilities, you usually set with a single database vendor in practice. If you suddenly start to switch databases in the middle of the project, something needs to be fixed with the process design, not database.

Re: PostgreSQL is enough

#108
post #91

Earlier quoted context omitted.

Well a database is just a binary file format with an API attached on. Because having hundreds or thousands of concurrent file handles across a data center is kind of hard.

ok I get that, but at least our applications usually only have one thing (one thread of one process on one machine) operating on the database. So maybe it's just our silly usecase

[deleted]

Re: PostgreSQL is enough

#109
post #91

Earlier quoted context omitted.

Well a database is just a binary file format with an API attached on. Because having hundreds or thousands of concurrent file handles across a data center is kind of hard.

ok I get that, but at least our applications usually only have one thing (one thread of one process on one machine) operating on the database. So maybe it's just our silly usecase

It's a narrow use-case that doesn't cover RDMS. SQLite fits it though.

Re: PostgreSQL is enough

#110
post #106

As a hardcore c++ guy, I recently switched to a company heavily into databases. I never had contact to databases before. And I'd like to go one step further: Why databases? I come from an industry that heavily uses custom binary file formats. And I'm still bewildered by the world of databases. They seem to solve many issues on the surface, but not really in pratice. The heavy limitations on data types, the update dis…

> They seem to solve many issues on the surface, but not really in practice I believe you haven't had a chance to work on problems that require an actual database. Multi-user access, ACID support, unified API (odbc/jdbc), common query language... all of these would require many man-years to set properly with a custom solution. > the update disasters, the incompatibility between different SQL engines etc all make it s…

All these comments seem to fuel my suspicion that we in fact shouldn't use databases, because we don't use any of these features. We just use them as external data storage for a single application. And not even that much data, like But the updating I would have expected to go more smoothly. If you make a point of using a software dedicated to managing data, I sure as hell would expect an update to go so smooth that I don't have to worry about or even notice it. In reality updates more often than not seem to come with undocumented errors. That is a constant source of frustration for me.
Post reply on HN