Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL 14

postgresql.org

Post reply on HN
Print view View original

161–170 of 293 posts

Re: PostgreSQL 14

#161

Post by why-el »

brightball wrote: post #159
skrebbel wrote: post #63

These changes look fantastic. If I may hijack the thread with some more general complaints though, I wish the Postgres team would someday prioritize migration. Like make it easier to make all kinds of DB changes on a live DB, make it easier to upgrade between postgres versions with zero (or low) downtime, etc etc. Warnings when the migration you're about to do is likely to take ages because for some reason it's going…

There’s a Ruby gem called strong_migrations that does this. It’s fantastic and I include it on all of my Ruby projects.

The gem does not solve these issues, merely tells you about them (and even then it can't catch all of them, only the ones that Rails defines).

Re: PostgreSQL 14

#162

Post by bananaoomarang »

jeremy_sikich wrote: post #71
buro9 wrote: post #23

Suppose I had a "friend" with a PostgreSQL 9.6 instance (a large single node)... what's the best way to upgrade to PostgreSQL 14?

Pglogical, replicate to a new database server. There will be minimal downtime and it allows you to test the process thoroughly.

Yeah this is how we do it... Even with terrabytes of data it seems to be pretty efficient and robust, and easy to just keep up the replicating version until you're satisfied it all looks good.

Re: PostgreSQL 14

#163

Post by tomhallett »

gunnarmorling wrote: post #121
tomhallett wrote: post #110

Earlier quoted context omitted.

Interesting that you bring this up. I looked into heroku's streaming connectors to facilitate an integration with materialize.com, but Heroku's support team wasn't confident we could sync all 187 postgres tables under 1 connection. I thought about using Debezium and Kafka to roll my own micro-batch ETL solution, but listening to this podcast made me walk away slowly: https://www.dataengineeringpodcast.com/datacoral-c…

Interesting, what was it from that podcast that made you reconsider? Always eager to learn about opportunities for improving the experience of using Debezium. Disclaimer: I work on Debezium

Oh wow, by "work on" you mean "the core maintainer of". Thank you for replying. :)

The main part I reconsidered based on was the level of effort taking the data from kafka and landing into snowflake, especially around handle postgres schema changes safely. I also have no experience with kafka, so I'd be out of my depth's pretty quickly for a critical part of the architecture. He also expressed the need for building quality checks into the kafka to snowflake code, but those details were a bit sparse (if i recall correctly).

Note: all of the above are probably outside the scope of debezium. :)

Note 2: your article [1] on using cdc to build audit logs w/ a "transactions" table blew my mind. Once I listened to your data engineering podcast interview [2], I knew there was some implementation of "event sourcing lite w/ a crud app" possible, so I was excited to see you had already laid it out.

1) https://debezium.io/blog/2019/10/01/audit-logs-with-change-d...

2) https://www.dataengineeringpodcast.com/debezium-change-data-...

Re: PostgreSQL 14

#164

Post by skrebbel »

jeltz wrote: post #98
skrebbel wrote: post #63

These changes look fantastic. If I may hijack the thread with some more general complaints though, I wish the Postgres team would someday prioritize migration. Like make it easier to make all kinds of DB changes on a live DB, make it easier to upgrade between postgres versions with zero (or low) downtime, etc etc. Warnings when the migration you're about to do is likely to take ages because for some reason it's going…

Migrations are quite highly prioritized in PostgreSQL. PostgreSQL has the by far best migration support of any database I have worked with. There is of course a lot of work left to do but that is true for many other areas in PostgreSQL. Also I can't agree at all with "nobody coded it", patches in this area generally welcome. My first real patch for PostgreSQL was about improving migrations. And there has been talk ab…

Sorry, I didn't mean offense. What I meant with "nobody coded it" is that the migration DX features that don't exist yet, likely don't exist simply because they haven't been made yet (and not because eg they're architecturally impossible or because the postgres team are stupid or sth).

Its hard to complain about OSS without attacking the creators, I tried to do that right buy clearly I failed nevertheless :-) Thanks for your contributions!

Re: PostgreSQL 14

#165

Post by simonebrunozzi »

Here we are, at a fantastic version 14, and still no sign of an MySQL AB-like company able to provide support and extensions to a great piece of open source software. There's a few small ones, yes, but nothing at the billion dollar size.

I am still unable to understand why.

Re: PostgreSQL 14

#166

Post by rathboma »

cmg wrote: post #122
rathboma wrote: post #75

Earlier quoted context omitted.

Beekeeper Studio maintainer here. It's 100% FOSS and cross platform (Windows, MacOS, Linux). I love Postgres and started Beekeeper because I wanted 'Sequel Pro for PSQL on Linux'. We have a lot of users so seems like we're doing something right. Very happy for new PSQL. If you have questions, let me know!

Thank you for your work on it! I used Sequel Pro for a long time for MySQL databases and occasionally use its successor Sequel Ace as well, but I'm moving my workflow to Beekeeper Studio as much as I can.

Glad to hear! Please file issues if anything is confusing or missing, it's a pretty friendly GitHub community.

Re: PostgreSQL 14

#167

Post by laurent92 »

bastawhiz wrote: post #99
tomhallett wrote: post #96

Earlier quoted context omitted.

An orthogonal migration issue which I'm hitting right now: we need to migrate from heroku postgres to aws rds postgres, and I'm stressed about the risk and potential downtime in doing so. If there was a way to make a replica in rds based on heroku, promote the rds replica to be the primary, hard switch our apps over to rds, that'd be a lifesaver. I'm working through this blog post [1] now, but there is still a bit to…

I did that exact migration. Unfortunately, to my knowledge, there's no way to do it with zero downtime. You need to make your app read only until the RDS instance has ingested your data, then you can cut over. For me, that was roughly one gigabyte of data and took about forty seconds. My best advice is to automate the whole thing. You can automate it with the Heroku and AWS CLIs. Test on your staging site until you c…

So, basically, Postgres would have a replication port which can be used for both replication/clustering and transfer across cloud providers. And sharding.

Re: PostgreSQL 14

#169

Post by TOMDM »

gavinray wrote: post #136
TOMDM wrote: post #127

Earlier quoted context omitted.

ZomboDB has always seemed really interesting to me. Elastics search, with PostgreSQL's everything else sounds like a perfect match, but I've never seen it used anywhere which is kind of spooky to me. Do you know of any non-toy use cases I can look at?

You can ask Eric Ridge on the ZomboDB Discord, he's really nice: https://discord.gg/hPb93Y9 Also probably one of the most knowledgeable people about both Postgres and Rust I've ever met. He makes his living primarily from supporting enterprises using ZomboDB (I think), not sure how much he can say, but it's worth asking.

You're not kidding, they were incredibly friendly and helpful.

Makes me wish I had a big deployment so I could justify sponsoring them.

I paid in cat pictures for now.

Re: PostgreSQL 14

#170

Post by znpy »

skrebbel wrote: post #63

These changes look fantastic. If I may hijack the thread with some more general complaints though, I wish the Postgres team would someday prioritize migration. Like make it easier to make all kinds of DB changes on a live DB, make it easier to upgrade between postgres versions with zero (or low) downtime, etc etc. Warnings when the migration you're about to do is likely to take ages because for some reason it's going…

My complain would be that there is no standard multi-master solution for postures, whereas mysql now has group replication as a native multi-master solution.
Post reply on HN