Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL for Everything

raphaelbauer.com

Post reply on HN
Print view View original

251–260 of 286 posts

Re: PostgreSQL for Everything

#251

Post by slfnflctd »

ethbr1 wrote: post #121
cyh555 wrote: post #105

Earlier quoted context omitted.

100% except when Kafka goes wrong, who maintains it?

There are two sizes of companies: those that can afford '1+ dedicated ____-person' and those that can't. Which should filter through to technology choices more than it does.

Or, you can do like one of my former bosses, and just rattle off a list of 60+ major projects which would require a team of 10 to make any reasonable progress on in the near future, then pin it on one underqualified person, refuse to provide a proper budget, and continually press them about why targeted deadlines are being missed.

Re: PostgreSQL for Everything

#252

Post by bbkane »

groundzeros2015 wrote: post #219
bbkane wrote: post #218

Earlier quoted context omitted.

What don't you like about SQLite views or foreign keys?

For example you can’t insert or update a SQLite view. I don’t even think SQLite lets you fully update a table schema. If you start trying to use it for sql and not just storing rows you run into these everywhere. SQLite is not serving the same needs as Postgres.

You can use a trigger with INSTEAD OF to update a view:

CREATE TRIGGER update_customer_emails_trigger INSTEAD OF UPDATE ON customer_emails BEGIN UPDATE customers SET email = new.email, name = new.name WHERE id = old.id; END;

Maybe that's not as flexible as you need.

SQLites process for table updates can be pretty onerous if their ALTER TABLE lacks support. Its a 12 step process the docs have the temerity to call "simple" instead of "tedious and risky" - https://www.sqlite.org/lang_altertable.html#otheralter

Re: PostgreSQL for Everything

#253

Post by Tostino »

ericpauley wrote: post #57
molf wrote: post #40

Earlier quoted context omitted.

I don't think the point is that PostgreSQL is great for everything. But you may get by with a single piece of infrastructure instead of 7. In most of the applications we build or maintain we use PostgreSQL + cloud storage. That's it. And it works very well, also for: storing JSON, full text search, as a queue, as a vector database. Other software may be better at providing those features, but I'm extremely happy we o…

The article says verbatim “PostgreSQL Replaces Clickhouse”. Coming from storing billions of rows in Clickhouse and performing dozens of materialized operations I shudder to think about what that would look like in a DB that doesn’t even support declarative IVM.

I've been working on that in Postgres for the past...way too long now: https://commitfest.postgresql.org/patch/6305/

Still not totally declarative, but it at least gets IVM into core in a way people can make use of.

Re: PostgreSQL for Everything

#254

Post by DC-3 »

Thaxll wrote: post #74
replwoacause wrote: post #7

I use SQLite for everything, and I'm perfectly happy with it. I'm aware of the concurrent writer issues, but at my scale it doesn't even matter.

The main issue with SQLite is the very poor type system, after testing it for an app I was shocked.

You can mitigate this with decent bindings, but I agree it's not ideal.

Re: PostgreSQL for Everything

#255

Post by sgarland »

sgt wrote: post #220
Tostino wrote: post #181

Earlier quoted context omitted.

Listen to this advice. I had a system that has ~600gb of blob data in bytea that could have easily been an S3 bucket + db reference. It made backups way more of a pain than necessary. It was intentional in the design, because I wanted total consistency with a single backup for the system. It worked great for years. But as we got more and more clients, it really should have been migrated to the above design to make su…

So the original advice actually still stands. You can still start off with Posgres, store it in BYTEA columns, and then work on a plan to use an object storage as you grow. S3 may not be possible and you will be evaluating other options like Minio

I guess I just don’t see the point for most applications. If you’re using a DBaaS, as most are, object storage from the same provider is almost certainly going to be cheaper, and it’s a trivial amount of code to handle the separate push / pull code.

Re: PostgreSQL for Everything

#256

Post by rco8786 »

The related posts at the end are...interesting.

> Three Strike Dismissal in One-On-Ones

> Keith Rabois recommends to dismiss your report when you feel bad about an upcoming one on one more than three times in a row. Interesting. …

Re: PostgreSQL for Everything

#257

Post by jroseattle »

> Events, queues and persistent logs are getting more and more important in today’s software systems. Systems like Kafka, RabbitMQ, SQS and others provide that functionality. But maintaining them is annoying, custom and you need the skillset.

In tech stack choices, I prefer staying simple as long as feasible. That said, you also need to know and understand concepts at a thorough level.

The above comment from the article suggests PG as a central server that simplifies event architecture. As if the "annoying, custom and needed skillset" around those specific alternatives are unnecessary baggage.

If you know anything about queues, scaling, availability, access semantics, message formats and concepts such as delivery guarantees, you find out very quickly that the server which stores a queued message is not the high-order bit in that equation.

Re: PostgreSQL for Everything

#258

Post by sanderjd »

jroseattle wrote: post #257

> Events, queues and persistent logs are getting more and more important in today’s software systems. Systems like Kafka, RabbitMQ, SQS and others provide that functionality. But maintaining them is annoying, custom and you need the skillset. In tech stack choices, I prefer staying simple as long as feasible. That said, you also need to know and understand concepts at a thorough level. The above comment from the arti…

To me, your concluding sentence cuts the opposite direction. The server is not the most important thing, and each new kind of server that exists in the system is an appreciable increase in maintenance burden. To me, taken together, this is an argument for waiting until you have a very concrete forcing function to introduce that new kind of server for this purpose.

I think a good way to think of this is: What empirical metric will the introduction of kafka (or whatever you choose) move in a positive direction? Latency? Oncall burden? The amount of code you have to maintain? Do you have correctness or data integrity metrics that this change would register on? etc.

This isn't at all intended as an unanswerable question. Many or most organizations will easily say "yes" that they expect some improvement on some metrics by adopting the "right" system for the job. But lots of other organizations are cargo culting "well we know this is the right way, so we should do it this way" long before any metrics they care about would demonstrate the improvement.

Re: PostgreSQL for Everything

#259

Post by CodesInChaos »

erispoe wrote: post #243
ethbr1 wrote: post #121

Earlier quoted context omitted.

There are two sizes of companies: those that can afford '1+ dedicated ____-person' and those that can't. Which should filter through to technology choices more than it does.

It's not 1+ person when a system needs to operate 24/7. To have a proper on-call rotation you need 5 to 8 people.

One specialist and a group of generalists is often enough. Especially if you are allowed to contact the specialist outside normal working hours in rare emergencies.

Re: PostgreSQL for Everything

#260

Post by groundzeros2015 »

throwawaythekey wrote: post #222
groundzeros2015 wrote: post #215

Earlier quoted context omitted.

> You might want to reduce costs with lambdas and DynamoDB. I don't think that's ever saved money. > because if they had that experience, they would have never written this article. That's not true.

> I don't think that's ever saved money. I spent about a year as a consultant in the AWS space, visited about ~15 clients of varying sizes. More often than not there's a single pg aurora instance responsible for 50%+ of the bill. Even worse are the serverless aurora offenders. All the indexes and guarantees of PG don't come cheaply and dynamodb pricing is not cheap but comparatively reasonable. It really is a good pr…

> All the indexes and guarantees of PG don't come cheaply

I’m currently paying digital ocean about 30 dollars a month for 4 commercial websites with fully featured Postgres.

Post reply on HN