Live data from Hacker News

PostgreSQL for Everything

raphaelbauer.com

41–50 of 286 posts

Re: PostgreSQL for Everything

#42
post #33

I tend to agree with quite a few points in the article, but some topics warrant some careful scrutiny. * As a message queue: Only if your required features are very basic, like if you need cluster communication and run your own coordination protocol on top. * High Volume Time Series: TimeScale works, but composes badly with other workloads on the same DB server ( from an operational perspective at scale ) * Vector Da…

Also to note: (Not a fault of PGVector again just a limit of our algorithmic knowledge) PGVector does HSNW or IVFlat indices ... (there is nothing better persistent) however it breaks down with high latency at LARGE amounts of vectors ( 100MIO+ ) that seems like a high ceiling, but when designing production RAG systems, you tend to do per chunk embeddings, or even visual patch embeddings... e.g one page of a document becomes 1024 vectors in itself (for visual patch embeddings ) ... so you hit those limits at 100000 pages already.. something larger organizations definitly have.

Re: PostgreSQL for Everything

#43
post #24

SQLite has so many advantages over PostgreSQL. No deamon. Single file per DB. Less configuration overhead.

It's probably perfect for a majority of work (and DuckDB takes that even further).

But for big, multi-writer work PG is the way to go.

Re: PostgreSQL for Everything

#45
post #24

SQLite has so many advantages over PostgreSQL. No deamon. Single file per DB. Less configuration overhead.

Postgres also has many advantages over SQLite.

Supporting more than 1 writer per process. Strict typing. Access controls. Replication at scale is more effecient than copy-pasting files (seems SQLite has improved on this one).

Re: PostgreSQL for Everything

#46
post #30

This kind of post (Postgres! It's all you need!) is getting pretty tiresome. Postgres does not even come close to a full replacement for Elastic, and that's just the first bullet. Looking down the list it is pretty easy to go: Yes, postgres can be used instead of that for extremely basic use cases, but it all goes out the window you actually need any of the power of these other tools.

I'm partial to Typesense, especially for smaller data sets, since it runs primarily in memory, is easy to use and is hella fast. For bigger data sets, I hear good things about Meilisearch.

Re: PostgreSQL for Everything

#47
post #30

This kind of post (Postgres! It's all you need!) is getting pretty tiresome. Postgres does not even come close to a full replacement for Elastic, and that's just the first bullet. Looking down the list it is pretty easy to go: Yes, postgres can be used instead of that for extremely basic use cases, but it all goes out the window you actually need any of the power of these other tools.

Fwiw, I, as someone who has worked on Postgres for a long time, also find it quite tiresome. Like there's plenty stuff I wouldn't use Postgres for, and I can probably get get more out of it than most.

Re: PostgreSQL for Everything

#48
post #39
post #33

I tend to agree with quite a few points in the article, but some topics warrant some careful scrutiny. * As a message queue: Only if your required features are very basic, like if you need cluster communication and run your own coordination protocol on top. * High Volume Time Series: TimeScale works, but composes badly with other workloads on the same DB server ( from an operational perspective at scale ) * Vector Da…

I like to consider Postgres the starting point for all of these things, that can be outgrown and replaced when appropriate. I do love just shoving everything in Postgres and seeing that I only end up needing a few additional dedicated services as the product groups. Redis is usually the next pickup for me.

Sure, thats a good way of working.. I just have the experience when handing over a project ( consulting ) anything i have put in place will never get replaced or kept for too long outgrowing its capacity by far, and offset with huge expenses in hardware or operations. Technically not my problem anymore ( except when it breaks on a maintenance contract ) but i still like to avoid it early if i can

Re: PostgreSQL for Everything

#49
post #8

sqlite for everything NVMe drives + Litestream + object storage(S3/R2..). sqlite simplifies things for the entire long tail of apps/services that aren't the Ubers and AirBNBs of the world.

It looks interesting, but deciding on where to put object storage is what keeps me from doing this. I don’t have an AWS or Cloudflare account and I’m not sure what to commit to.

Also, apparently Litestream could use a filesystem instead of an object store?

Re: PostgreSQL for Everything

#50

Earlier quoted context omitted.

MySQL was a proven solution back in the day, i.e late 2000s. Github/Twitter/Heroku etc were using it. In the past 10-15 years, Postgres has come a long way.

MySQL was always behind in terms of features. In early 2000s it had very limited constraints. Most people were running in ISAM backend and did not even have transaction support. It was being used by people who did not understand how advanced relational DBs were being used. What changed is Postgres overtook its actual competition, which were Oracle, SQL Server and Sybase. MySQL caught up as well as far as I know, but…

In the early days, PostgreSQL was so much more awkward to deal with. Crash-prone at first, and then there was the whole business around having to drop the db during upgrades. It didn't really match MySQL operationally until around 2002.

During the dot com era it was common to develop and launch on MySQL with the intention of migrating to something else if they became successful (though your typical LAMP stack developer regarded Oracle and SQL Server as being deeply 'weird', so many were willing to stick with MySQL despite the well-known limitations of MyISAM).

From where I'm standing, it seems that PostgreSQL became clearly preferable for new projects from the mid 2000s onwards, but it was only the Oracle acquisition that began to push existing users off MySQL.

Post reply on HN