Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

WAL-RUS: a Rust Rewrite of WAL-G for PostgreSQL Backups

clickhouse.com

Post reply on HN
Print view View original

11–20 of 23 posts

Re: WAL-RUS: a Rust Rewrite of WAL-G for PostgreSQL Backups

#11

Post by __s »

sgt wrote: post #10
__s wrote: post #9

Earlier quoted context omitted.

indeed, wal-g actually started as a port of wal-e which was Python: https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-... wal-g was a much larger improvement over wal-e. we're optimizing the margins here

I'd like to use the one getting the most community support. So too soon to wait for Rust vs Go. Although on paper, Rust is better.

tbf it took 4 years since PG15 support was added for me to fix remote BASE_BACKUP support & wal-g base backups being inconsistent on PG15+ (parameter typo had pg_backup_stop return before wal archived far enough for consistency)

https://github.com/wal-g/wal-g/pull/2262

but yes, this is young project, so fair take

Re: WAL-RUS: a Rust Rewrite of WAL-G for PostgreSQL Backups

#12

Post by saisrirampur »

__s wrote: post #9
nasretdinov wrote: post #5

I must say I'm quite pleased to see how well Go version works. It does only use 1.5x the CPU and (predictably) much more RAM/VRAM, but not a crazy amount either (the expected increase is 2x). Of course you can write a more optimal version in C / C++ / Zig / Rust, but at the same time Go is much easier to write and you don't pay for the convenience with an absurd performance loss like in Python or PHP

indeed, wal-g actually started as a port of wal-e which was Python: https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-... wal-g was a much larger improvement over wal-e. we're optimizing the margins here

++ We’re big Go fans, most of PeerDB is written in Go: https://github.com/PeerDB-io/peerdb

The importance of optimizing (resource) margins and having predictable memory usage increases significantly in the DBaaS/Postgres world, where your process coexists and competes with other critical workloads.

Also, WAL-RUS isn’t rocket science. Postgres already exposes a bunch native constructs for WAL archival, making development fairly straightforward even in Rust.

TL;DR: when to choose Rust or Go really depends on the workload and what you are going after.

Re: WAL-RUS: a Rust Rewrite of WAL-G for PostgreSQL Backups

#13

Post by nasretdinov »

__s wrote: post #8
valentynkit wrote: post #6

Quick one on the benchmark: was the 2.8GB peak virtual or resident? Go reserves a large virtual arena it mostly never faults in, so RSS tends to be a fraction of the virtual peak, and if Postgres headroom was getting squeezed off the virtual number you were sizing against memory the kernel never actually charges for.

Correct. We tune overcommit so postgres reliably returns out of memory. It becomes complicated to accurately tune overcommit for every AWS instance type. We configure GOMEMLIMIT/cgroups but those are about RSS. Outliers come together: instances running queries out of memory on our service tend to also be pushing other resource limits, causing wal-g & prometheus exporters to start having more erratic memory usage at t…

You probably could limit the bloating of Go programs by setting GOMAXPROCS to something like 1 or 2 on smaller machines, but then again you wouldn't get the best performance. So IMO good call here to rewrite it in a language without GC.

Re: WAL-RUS: a Rust Rewrite of WAL-G for PostgreSQL Backups

#17

Post by cipherself »

The article’s distinction about memory is a bit too simplified. Virtual memory is address space mapped or reserved by the process. RSS is resident physical pages, not necessarily “actively used.”. RSS also overcounts shared memory, which is why PSS/USS matter.

Re: WAL-RUS: a Rust Rewrite of WAL-G for PostgreSQL Backups

#18

Post by __s »

whitepoplar wrote: post #15

As someone who only has a cursory knowledge of Postgres backup systems, how does this compare to something like pgBackRest? When would someone reach for one over the other?

If you're running pg yourself I recommend pgbackrest. It doesn't run as a daemon, & it forks multiple processes for concurrency. But it's simple to run as archive_command & is light on resources outside concurrency

wal-g/wal-rus have higher throughput

Re: WAL-RUS: a Rust Rewrite of WAL-G for PostgreSQL Backups

#19

Post by keynha »

saisrirampur wrote: post #12
__s wrote: post #9

Earlier quoted context omitted.

indeed, wal-g actually started as a port of wal-e which was Python: https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-... wal-g was a much larger improvement over wal-e. we're optimizing the margins here

++ We’re big Go fans, most of PeerDB is written in Go: https://github.com/PeerDB-io/peerdb The importance of optimizing (resource) margins and having predictable memory usage increases significantly in the DBaaS/Postgres world, where your process coexists and competes with other critical workloads. Also, WAL-RUS isn’t rocket science. Postgres already exposes a bunch native constructs for WAL archival, making developm…

That coexistence is also why the GC-free rewrite helps more than the speed numbers suggest. An archiver is allocation-light until it hits a compression burst, then the Go heap can spike toward 2x live right when Postgres wants that memory. GOMEMLIMIT caps the spike but pays in GC CPU during exactly those bursts, so on a small instance, you are trading OOM risk for throughput. Rust removes that dial.

Re: WAL-RUS: a Rust Rewrite of WAL-G for PostgreSQL Backups

#20

Post by pjmlp »

nasretdinov wrote: post #5

I must say I'm quite pleased to see how well Go version works. It does only use 1.5x the CPU and (predictably) much more RAM/VRAM, but not a crazy amount either (the expected increase is 2x). Of course you can write a more optimal version in C / C++ / Zig / Rust, but at the same time Go is much easier to write and you don't pay for the convenience with an absurd performance loss like in Python or PHP

In many cases it could even be more optimised, but somehow using value types is a lost art.

Granted Go could make it much more easier to understand when they escape the stack into the heap, as many of their design decisions, simple and easy aren't the same.

Post reply on HN