151–160 of 323 posts
My Trifecta is: Postgres, Redis, S3 Hasn't steered my wrong yet. Every once in a while I'm tempted to try to use Postgres for Pub/Sub but then I realize that I need Redis for caching and sidekiq anyways, and Redis is amazing too, so why bother.
Earlier quoted context omitted.
My saying has always been: be nice to the DB Don't use it anymore than you have to for your application. Other than network IO it's the slowest part of your stack.
Handling business logic in the database is often going to be an order of magnitude faster than the application layer of some of the popular language stacks (looking at you, Rails, Node, etc). It also will outlive whatever webstack of the day (and acquisition which of en requires a re-write of the application layer but keeps general database structure - been there done that).
Earlier quoted context omitted.
I’m with you in general, but what about vector search? It really feels like the DB industry has taken a huge step backward from the promise of SQL. Switching from Postgres to SQLite is easy because the underlying queries are at least similar. But as soon as you introduce embeddings, every system is totally different (and often changing rapidly).
Just use SQLite? Specialized vector indexes become important when you have a large number of vectors, but the reality of software is that it is unlikely that your application will ever be used at all, let alone reach a scale where you start to hurt. Computers are really fast. You can go a long way with not-perfectly-optimized solutions. Once you have proven that users actually want to use your product and see growth…
There are plenty of proprietary databases and APIs, but now you're taking on a dependency and assuming a certain amount of risk.
Postgres is enough as long as you have a good multi-tenant setup e.g. a separate database per customer. Ran a single postgres instance with multi-tenant SaaS product that crossed 4B records in a few tables, even with partitions and all the optimization in the world, it still hurts to have one massive database. We still got bought tho, so I will agree its enough
Was talking to coworker yesterday about a spectrum of where code lives, and the differences from where I started to where I am now in understanding. Start after college and backend web dev was fully in scripting language, Python or Ruby, and ORMs that completely fogged where any of the data was stored. Rails and ActiveRecord is so good at shrouding the database to the point where you type commands that create databas…
I'm one of the makers of ParadeDB, a modern alternative to Elasticsearch. We build Postgres extensions to do fast search (pg_bm25) and analytics (pg_analytics). I love Postgres. If you have a small workload, like a startup, it certainly makes sense to stay within Postgres as long as you can. The problem is, at scale, Postgres isn't the answer to everything. Each of the workloads one can put in Postgres start to grow…
In principle, seems like it should work to allow large scale distribution across many servers. But the actual management of replicas and deciding which servers to place partitions, redistributing when new servers are added, etc. could lead to a massive amount of operational overhead.
Earlier quoted context omitted.
> [...] sqlite is the 80% case and is also dead simple to get going and genuinely performant. I don't understand this. PostgreSQL is ALSO dead simple to get going, either locally or in production. Why not just start off at 90%? I mean, I get there are a lot of use cases where sqlite is the better choice (and I've used sqlite multiple times over the years, including in my most recent gig), but why in general?
> PostgreSQL is ALSO dead simple to get going I'm not saying it's hard to set up Postgres locally, but sqlite is a single binary with almost no dependencies and no config, easily buildable from source for every platform you can think of. You can grab a single file from sqlite.org, and you're all set. Setting up Postgres is much more complicated in comparison (while still pretty simple in absolute terms - but starting…
Could you write an OS based on PostgreSQL?:)
Specifically with PostgreSQL, number of connections is still the killer (if you don’t have smart proxies). So you can’t be cavalier about putting as many use cases as possible. For example, when using PG pub/sub, you will run out of connections quick. Generally, all DBMS needs a smart self adjusting query killer. Without it, one bad query will ruin it for everyone.
Supavisor connection pooler: https://github.com/supabase/supavisor
I'm one of the makers of ParadeDB, a modern alternative to Elasticsearch. We build Postgres extensions to do fast search (pg_bm25) and analytics (pg_analytics). I love Postgres. If you have a small workload, like a startup, it certainly makes sense to stay within Postgres as long as you can. The problem is, at scale, Postgres isn't the answer to everything. Each of the workloads one can put in Postgres start to grow…
what is "at scale"? Is there a specific metric or range of metrics that raises a flag to begin considering something else? For example, in the olden days when it was my problem, page load times were the metric. Once it got high enough you looked for the bottleneck, solved it, and waited. When the threshold was broken again you re-ran the same process. Is there an equivalent for postgres?
I don't know that there is a canonical solution for scaling Postgres data for a single database across an arbitrary number of servers.
I know there is CockroachDB which scales almost limitlessly, and supports Postgres client protocol, so you can call it from any language that has a Postgres client library.