31–40 of 66 posts
Will WAL-G eventually support the same archive targets as WAL-E (S3 and work-alikes, Azure Blob Store, Google Storage, Swift, File System)?
Earlier quoted context omitted.
Any plans to support backup to Google Cloud Storage instead of just S3?
Or some sort of pluggable storage system. For now, since I'm also on GCP, I'm using PGHoard: https://github.com/ohmu/pghoard
This is great. Can't wait to be using it. We've been using WAL-E for years and this looks like a big improvement. The steady, high throughput is a big deal – our prod base backups take 36 hours to restore, so if the recovery speed improvements are as advertised, that's a big win. In the kind of situation in which we'd be using these, the difference between 9 hours and 36 hours is major. Also, the quality of life impr…
> our prod base backups take 36 hours to restore, so if the recovery speed improvements are as advertised, that's a big win.
Yes, if you attach 16TB of storage to each instance, your back-up restores may take a while. :))
Hello everyone, I'm the primary author for WAL-G and would be happy to answer any questions.
It would be nice if you could sign the tarball with the binary. I see the tag is already signed so hopefully it's not much trouble: https://wiki.debian.org/Creating%20signed%20GitHub%20release...
Earlier quoted context omitted.
WAL-G is not yet production ready, but it has been used in a staging environment for the past few weeks without any issues. Once fdr adds parallel WAL support, he plans to take it into production.
Cool cool. is google cloud storage on the roadmap?
Wow, great work! I am definitely going to test this out over the weekend. However AFAICT the `aws.Config` approach breaks certain backwards compatibility w/how wal-e handles credentials. Also wal-g does not currently support encryption. FWIW, I would love to simply drop-in wal-g without having to make any configuration changes.
Hello everyone, I'm the primary author for WAL-G and would be happy to answer any questions.
Do you have plans to support encrypted backups?
I've used WAL-E (the predecessor of this) for backing up Postgres's DB for years and it's been a very pleasant experience. From what I've read so far this looks like it's superior in every way. Lower resource usage, faster operation, and the switch to Go for WAL-G (v.s. Python for WAL-E) means no more mucking with Python versions either. Great job to everybody that's working on this. I'm looking forward to trying it…
i have a huge ETL pipeline at twitch that relies heavily on wal-e, and installs it on worker nodes and things like that.
that being said, if WAL-G is faster, i don't care waht it is written in, and am happy to use it
Hello everyone, I'm the primary author for WAL-G and would be happy to answer any questions.
Hello everyone, I'm the primary author for WAL-G and would be happy to answer any questions.
Thanks! Does WAL-G provides some kind of "continuous backup" where changes committed to the database are continuously streamed to the backup storage? Or does it work "step by step", for example by backing up every 5 minutes or every 10 MB?
Both back up PG's WAL files (Write Ahead Log) and allow restoring your database state as it was at a specific time or after a specific transaction committed. This is known as point-in-time recovery (PITR) [0]
Users and admins make mistakes, and accidentally delete or overwrite data. With PITR you can restore in a new environment, just before the mistake occurred and recover the data from there.
[0] https://www.postgresql.org/docs/9.6/static/continuous-archiv...