Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Barman – Backup and Recovery Manager for PostgreSQL

github.com

Post reply on HN
Print view View original

11–20 of 29 posts

Re: Barman – Backup and Recovery Manager for PostgreSQL

#11

Post by the_angry_angel »

hephaes7us wrote: post #8
levkk wrote: post #6

Last time I checked, Barman didn't support backups to S3. That's why (for us) pgBackRest was such a big deal: it could offload full and incremental backups to a basically limitless and reliable medium. I think (and I'm probably wrong now) that Barman only could push backups to another Linux machine (e.g., EC2 box), so you had to worry about your backup system _on top_ of the main DB. So I'm really hoping someone will…

Something like Rclone and a cron job, or else s3 mounted via FUSE, could possibly bridge that. Of course then you have to worry about reliability of the bridge...

There’s support via Barman cloud - we use it for azure at work but s3 and others are supported iirc

Re: Barman – Backup and Recovery Manager for PostgreSQL

#13

Post by hakube »

levkk wrote: post #6

Last time I checked, Barman didn't support backups to S3. That's why (for us) pgBackRest was such a big deal: it could offload full and incremental backups to a basically limitless and reliable medium. I think (and I'm probably wrong now) that Barman only could push backups to another Linux machine (e.g., EC2 box), so you had to worry about your backup system _on top_ of the main DB. So I'm really hoping someone will…

They have a plugin that supports backups to cloud storage

https://docs.pgbarman.org/release/3.12.0/user_guide/barman_c...

For CNPG: https://github.com/cloudnative-pg/plugin-barman-cloud

Re: Barman – Backup and Recovery Manager for PostgreSQL

#14

Post by Tostino »

subhobroto wrote: post #5
ninjaoxygen wrote: post #4

We use Barman inside Kubernetes via CloudNativePG's plugin, as it is the default backup plugin. Barman has always been solid for backup and restore, however configuring backup in CNPG is a little more interesting - WAL limits need to be set carefully or you just end up filling WAL volumes and the database becoming unavailable.

> We use Barman inside Kubernetes via CloudNativePG's plugin, as it is the default backup plugin. right and here's why CloudNativePG chose Barman over pgBackRest: https://github.com/cloudnative-pg/cloudnative-pg/issues/3077 > WAL limits need to be set carefully or you just end up filling WAL volumes and the database becoming unavailable. This is true. For anyone getting alarmed that this is due to a bug in PostgreSQL…

I thought that link was actually going to have a discussion on why they chose it. No such discussion exists.

Re: Barman – Backup and Recovery Manager for PostgreSQL

#15

Post by subhobroto »

Tostino wrote: post #14
subhobroto wrote: post #5

Earlier quoted context omitted.

> We use Barman inside Kubernetes via CloudNativePG's plugin, as it is the default backup plugin. right and here's why CloudNativePG chose Barman over pgBackRest: https://github.com/cloudnative-pg/cloudnative-pg/issues/3077 > WAL limits need to be set carefully or you just end up filling WAL volumes and the database becoming unavailable. This is true. For anyone getting alarmed that this is due to a bug in PostgreSQL…

I thought that link was actually going to have a discussion on why they chose it. No such discussion exists.

The way I read that issue and the linked discussion was, that pgBackRest handles a lot of details itself that's otherwise handled by Kubernetes. Hence, a lot of functionality in pgBackRest is not only redundant but incompatible with how Kubernetes CSI could be used to provide incremental and differential backups. Hence, Barman and `barman-cloud` plugins are a better, natural fit for a Kubernetes environment than pgBackRest.

Re: Barman – Backup and Recovery Manager for PostgreSQL

#16

Post by Eikon »

hephaes7us wrote: post #8
levkk wrote: post #6

Last time I checked, Barman didn't support backups to S3. That's why (for us) pgBackRest was such a big deal: it could offload full and incremental backups to a basically limitless and reliable medium. I think (and I'm probably wrong now) that Barman only could push backups to another Linux machine (e.g., EC2 box), so you had to worry about your backup system _on top_ of the main DB. So I'm really hoping someone will…

Something like Rclone and a cron job, or else s3 mounted via FUSE, could possibly bridge that. Of course then you have to worry about reliability of the bridge...

https://github.com/Barre/ZeroFS Should do a great job at this.

Re: Barman – Backup and Recovery Manager for PostgreSQL

#17

Post by bakies »

levkk wrote: post #6

Last time I checked, Barman didn't support backups to S3. That's why (for us) pgBackRest was such a big deal: it could offload full and incremental backups to a basically limitless and reliable medium. I think (and I'm probably wrong now) that Barman only could push backups to another Linux machine (e.g., EC2 box), so you had to worry about your backup system _on top_ of the main DB. So I'm really hoping someone will…

Huh... Opposite experience. Barman cloud (s3 backups) is the only way I've ever used it. I didn't realize it wasn't the only way. Makes sense it could just use a filesystem.

https://docs.pgbarman.org/release/3.14.1/user_guide/barman_c...

Re: Barman – Backup and Recovery Manager for PostgreSQL

#18

Post by timacles »

hephaes7us wrote: post #8
levkk wrote: post #6

Last time I checked, Barman didn't support backups to S3. That's why (for us) pgBackRest was such a big deal: it could offload full and incremental backups to a basically limitless and reliable medium. I think (and I'm probably wrong now) that Barman only could push backups to another Linux machine (e.g., EC2 box), so you had to worry about your backup system _on top_ of the main DB. So I'm really hoping someone will…

Something like Rclone and a cron job, or else s3 mounted via FUSE, could possibly bridge that. Of course then you have to worry about reliability of the bridge...

Mounting S3 with Fuse is not stable or performant enough at scale for backup storage

Re: Barman – Backup and Recovery Manager for PostgreSQL

#19

Post by philippemnoel »

We (ParadeDB) use Barman via CloudNativePG for almost all our deployments. It's been solid, although I've had a few complaints about 1) inability to set S3 storage classes, 2) slow upload for very large databases.

Nonetheless, very happy to see this project on the front page of HN!

Re: Barman – Backup and Recovery Manager for PostgreSQL

#20

Post by Synthetic7346 »

ninjaoxygen wrote: post #4

We use Barman inside Kubernetes via CloudNativePG's plugin, as it is the default backup plugin. Barman has always been solid for backup and restore, however configuring backup in CNPG is a little more interesting - WAL limits need to be set carefully or you just end up filling WAL volumes and the database becoming unavailable.

Can the plugin do non objectstore backups? E.g. If I don't want to use S3 / minio / whatever blobstore for my homelab
Post reply on HN