Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL for Everything

raphaelbauer.com

Post reply on HN
Print view View original

221–230 of 286 posts

Re: PostgreSQL for Everything

#221

Post by frollogaston »

andai wrote: post #140
psadauskas wrote: post #139

My general rule of thumb is "Use Postgres until you've discovered why you can't use Postgres." Anything you introduce is another moving part you have to operate and maintain, and in the beginning, Postgres can probably handle it. Wait for load, see where its failing, and then you'll have a better idea if adding another tool is worth the cost.

Doesn't the same argument apply even more to using SQLite instead?

Yes. I already did that once though, so I skip that step now (unless ofc it's a SQLite usecase).

Re: PostgreSQL for Everything

#222

Post by throwawaythekey »

groundzeros2015 wrote: post #215
ChicagoDave wrote: post #137

This is exactly how tightly coupled, unmaintainable software is constructed. By picking the tools before understanding the model and building bespoke architecture. You pick the tools that the business model requires. It might be a relational data store. It might not be. You might want an event store. You might want to reduce costs with lambdas and DynamoDB. You may need a pub/sub event broker. The OP clearly loves Po…

> You might want to reduce costs with lambdas and DynamoDB. I don't think that's ever saved money. > because if they had that experience, they would have never written this article. That's not true.

> I don't think that's ever saved money.

I spent about a year as a consultant in the AWS space, visited about ~15 clients of varying sizes.

More often than not there's a single pg aurora instance responsible for 50%+ of the bill. Even worse are the serverless aurora offenders.

All the indexes and guarantees of PG don't come cheaply and dynamodb pricing is not cheap but comparatively reasonable. It really is a good product if you know how to use it.

Re: PostgreSQL for Everything

#223

Post by frollogaston »

I use Postgres for a lot of things where textbooks say not to, but not caching. I'm not going to do it with triggers. Maybe if it supported TTL properly, even then, probably don't want to think about whether caching will bog down the rest of the DB.

Re: PostgreSQL for Everything

#224

Post by _flux »

Ozzie_osman wrote: post #9

I love postgres and use it heavily, but I still don't fully understand how it overlook MySQL. Maybe because of Heroku adopting it. MySQL was generally faster, and while MyISAM was a bit limited Innodb was pretty powerful, and you had the choice. It was also simpler (imo) and avoided a lot of the xid/vacuum issues. That said, still love Postgres. But at the time it started eclipsing MySQL, MySQL felt better positioned…

I always thought Postgres was the one that did correctness first, then performance, while MySQL was the inverse. I also enjoyed Postgres documentation. But in practice I have very little experience with MySQL, but I do recall it liked to silently coerce invalid dates and its UTF-8 wasn't quite UTF-8.

And MySQL apparently still doesn't support transactional DDL (i.e. BEGIN, ALTER, ALTER, UPDATE, COMMIT), which is quite nice for db schema version migrations.

Re: PostgreSQL for Everything

#226

Post by c0l0 »

HighlandSpring wrote: post #41

This isn't just theory either, for example: Revolut is a bank that does all its event persistence and streaming on top of postgres. No traditional message queues/brokers in their stack. https://medium.com/revolut/recording-more-events-but-where-w...

During pgConf.eu in 2016(-ish, could have been one or two years later; I don't remember too well), a representative of payment processor Adyen told the audience that they were, essentially, one big postgres cluster in their backend, too ("cluster" used as per the postgres-native meaning of the term, as in, an installation on a single host with a data directory containing any number of databases).

Re: PostgreSQL for Everything

#227

Post by renegat0x0 »

andai wrote: post #140
psadauskas wrote: post #139

My general rule of thumb is "Use Postgres until you've discovered why you can't use Postgres." Anything you introduce is another moving part you have to operate and maintain, and in the beginning, Postgres can probably handle it. Wait for load, see where its failing, and then you'll have a better idea if adding another tool is worth the cost.

Doesn't the same argument apply even more to using SQLite instead?

I used this approach to drive entire app, and it works. Nearly all data are fetched from SQLite. User can select a database, which can change app views, and the data. In my experience it is quite fast.

My example for android app:

https://f-droid.org/pl/packages/io.github.rumcajs.offlineweb...

Note that I am not android experienced programmer, and I am still learning.

Re: PostgreSQL for Everything

#228

Post by vjvjvjvjghv »

dorfsmay wrote: post #203
throwitaway222 wrote: post #197

Earlier quoted context omitted.

A counter anecdata. We transitioned from a postgres job queue to Rabbit. We had never ending problems after that, many of them were misunderstandings, some where just wrong-fit. We migrated because we had some time on our hands and thought we would alleviate some high pressure jobs. Not only did it not solve the problem, but having written all the code that decides when to pull the next message and what to do with it…

I tried very hard to use postgres as a queue, it was robust but slow once I started to push from more than a few processes/servers. Moving to zeromq initially and sqs after solved all my perf issues, and was still solid.

I think that’s the way to go. Start with Postgres and only if there are problems, then think about something more specialized. Same for microservices. Start simple and introduce a service when really needed.

I hate it when people already start out with 10 or more different systems/services for a few messages per second.

Re: PostgreSQL for Everything

#229

Post by TheCapeGreek »

Anecdotally:

The main caveat as someone who works on mostly average web CRUD apps, is that "Use PG/SQLite for everything" usually falls flat when the tools I use day to day don't support that use case super well or have rougher edges.

If your framework/ORM/whatever of choice doesn't support the full feature set of that driver compared to Redis/ES/Whatever you're replacing, you'll find yourself going down rabbit holes doing workarounds instead of staying with the "happy path" and just using separate tech for what it's specialised in.

If you already are doing most of these sorts of features by yourself instead of with frameworks, maybe it's fine, but this does start to feel like a time-to-release hindrance if you don't want to fiddle with the minutia.

Re: PostgreSQL for Everything

#230

Post by jankovicsandras »

cauchyk wrote: post #106

as someone who loves postgres, this take is getting pretty old. yes we can do quite a bit with extensions but extensions often need to interface with external systems and even then managed providers don't consistently support all extensions. some gaps: bm25 indexes, olap support, also extensions also run into licensing restrictions.

You can do BM25 and hybrid search in Postgres.

Shameless plug: https://github.com/jankovicsandras/plpgsql_bm25 BM25 search implemented in PL/pgSQL ( Unlicense / Public domain )

The repo includes also plpgsql_bm25rrf.sql : PL/pgSQL function for hybrid search ( plpgsql_bm25 + pgvector ) with Reciprocal Rank Fusion; and Jupyter notebook examples.

Post reply on HN