Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL is enough

gist.github.com

Post reply on HN
Print view View original

311–320 of 323 posts

Re: PostgreSQL is enough

#311

Post by johnthescott »

stickfigure wrote: post #83
brightball wrote: post #58

Earlier quoted context omitted.

PG works really well as a message queue and there's several excellent implementations on top of it. Most systems are still going to need Redis involved just as a coordinator for other pub/sub related work unless you're using a stack that can handle it some other way (looking at BEAM here). But there are always going to be scenarios as an application grows where you'll find a need to scale specific pieces. Otherwise t…

> PG works really well as a message queue It's also worth noting that by using PG as a message queue, you can do something that's nearly impossible with other queues - transactionally enqueue tasks with your database operations. This can dramatically simplify failure logic. On the other hand, it also means replacing your message queue with something more scalable is no longer a simple drop-in solution. But that's wor…

yep.

Re: PostgreSQL is enough

#312

Post by jauco »

bun_terminator wrote: post #225
dig1 wrote: post #128

Earlier quoted context omitted.

> because we don't use any of these features. We just use them as external data storage for a single application. You are using it :) Reboot the server where the database runs or suddenly cut off the connection. Unless you have ACID-compatible storage, you'll have malformed data. Plan for the future and use a database from the start. When your project/company expands and starts to use multiple applications/services (…

eh the oracle upgrades went awful. But I try to not touch the database at all if I can

Well, oracle is kind of a beast. If all you need is data storage and maybe an index for fast lookup within 10G of data, sqlite will probably do the trick. It is a library, not a server process so you compile it into your app and there is very little ops work to do. As a benefit, you can do many data access patterns that are closer to what you’d do if you wrote the data access code by hand (no network latency etc)

Re: PostgreSQL is enough

#313

Post by philihp »

TheCapeGreek wrote: post #30
superb-owl wrote: post #2

I often go down rabbit holes like this, trying to collapse and simplify the application stack. But inevitably, as an application grows in complexity, you start to realize _why_ there's a stack, rather than just a single technology to rule them all. Trying to cram everything into Postgres (or lambdas, or S3, or firebase, or whatever other tech you're trying to consolidate on) starts to get really uncomfortable. That s…

On top of that, a lot of discourse seems to happen with an assumption that you only make the tech/stack choice once. For the majority of apps, just doing basic CRUD with a handful of data types, is it that hard to just move to another DB? Especially if you're in framework land with an ORM that abstracts some of the differences, since your app code will largely stay the same.

The way you've trivialized a database vendor swap makes me curious how often you do this.

Re: PostgreSQL is enough

#314

Post by throwaway918274 »

cpursley wrote: post #151
throwaway918274 wrote: post #46

My Trifecta is: Postgres, Redis, S3 Hasn't steered my wrong yet. Every once in a while I'm tempted to try to use Postgres for Pub/Sub but then I realize that I need Redis for caching and sidekiq anyways, and Redis is amazing too, so why bother.

If you're open to Elixir (you'll like it coming from Ruby) then you don't even need Redis. Oban + Postgres for jobs, WalEx for database events, Nebulex for distributed caching. It simplifies things so much (and is cheaper to run).

I keep hearing about it, just never got around to actually doing anything with it...but I might have that itch now. I'll look into these things thanks.

Re: PostgreSQL is enough

#315

Post by throwaway918274 »

gen220 wrote: post #215
throwaway918274 wrote: post #46

My Trifecta is: Postgres, Redis, S3 Hasn't steered my wrong yet. Every once in a while I'm tempted to try to use Postgres for Pub/Sub but then I realize that I need Redis for caching and sidekiq anyways, and Redis is amazing too, so why bother.

Have you had to deal with read-heavy/OLAP queries? Of the kind that can't be cached effectively (i.e. arbitrary filters).

Our company has spend a lot of resources optimizing these kinds of queries in our application. It's been a bit of a struggle since we also use GraphQL, but we've been doing alright. Partial indexes and liberal use of jsonb have been doing a lot of heavy lifting for us. Our application is extremely read heavy, so the performance hit from updating lots of indexes hasn't been an issue for us (yet).

Re: PostgreSQL is enough

#317

Post by garfij »

bun_terminator wrote: post #226
DanielHB wrote: post #223

Earlier quoted context omitted.

Besides what others are saying here, data integrity is a lot easier with a relational database if your data is heavily relational. I have managed systems with About data types, what limitations are you referring to? jsonb is well supported in many dbs and throwing random large binary blobs in the middle with your normal data is a bad idea with or without a relational database. I can see the appeal to replace noSQL so…

I think my wtf moment was that you can't have real variable length strings. And I'm pretty pampered by c++ highly nuanced numeric types with varying sizes and signed/unsigned.

Other people have said it in this thread, but I think your experience is being severely clouded by Oracle. Postgres, MySql, and all have the `TEXT` datatype which are variable length (though with some upper bound)

Re: PostgreSQL is enough

#318

Post by realaleris149 »

themerone wrote: post #121
bun_terminator wrote: post #113

Earlier quoted context omitted.

The first thing I did was ask why we don't use sqlite or at least postgres. The answer was that they are free and therefore we don't trust them. So Oracle it is. Which is bananas because our customers have to buy an oracle license, which is money that we don't get. Pretty wild

Sqlite is running on billions of devices and has a test suite with 100% branch coverage. Only a fool would put more trust in Oracle.

> has a test suite with 100% branch coverage

Is actually even more than that, has 100% MC/DC coverage [1], which is something that is not easily achievable for most projects.

[1] https://www.sqlite.org/testing.html#:~:text=In%20this%20way%....

Re: PostgreSQL is enough

#319

Post by realaleris149 »

adlpz wrote: post #5

Love postgres and use it extensively. However, there's always an issue when I start doing the more advanced stuff: how do I combine that with all my years of experience with version control, code reviews, types, tests, static analysis and all the niceties of coding in general? Migrations?

There is a bit of tooling needed but is already around. For Java for example I had very good experience with a combination of flyway [1] for migrations, testcontainers [2] for making integration tests as easy as unit tests and querydsl [3] for a query and mapping layer.

[1] https://github.com/flyway/flyway

[2] https://java.testcontainers.org/modules/databases/postgres/

[3] https://github.com/querydsl/querydsl

Re: PostgreSQL is enough

#320

Post by realaleris149 »

vmfunction wrote: post #28
adlpz wrote: post #5

Love postgres and use it extensively. However, there's always an issue when I start doing the more advanced stuff: how do I combine that with all my years of experience with version control, code reviews, types, tests, static analysis and all the niceties of coding in general? Migrations?

If you want to serious develop postgres as how you develop code, then you should also use a LSP. https://github.com/supabase/postgres_lsp

If you are using IntelliJ (Ultimate), it also have very good support for SQL (syntax highlighting, autocomplete, etc.) and is Postgresql flavor aware.

https://www.jetbrains.com/help/idea/database-tool-window.htm...

Post reply on HN