Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
1–10 of 159 posts
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#2Is this new async. I/O feature for Linux only?
I know Windows has IOCP and also now an IORing implementation of its own (Less familiar with macOS capabilities other than POSIX AIO).
https://learn.microsoft.com/en-us/windows/win32/api/ioringap...
Update: Most of the comments below seem to be missing the fact that Windows now also has an IORing implementation, as I mentioned above. Comparison article here:
https://windows-internals.com/ioring-vs-io_uring-a-compariso...
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#3I recently deployed Postgres on a dedicated Hetzner EX-44 server (20 cores, 64GB RAM, 2x 512GB NVMe SSDs in RAID 1) for €39/month. The price-to-performance ratio is exceptional, providing enterprise-level capacity at a fraction of typical cloud costs.
For security, I implemented TailScale which adds only ~5ms of latency while completely eliminating public network exposure - a worthwhile tradeoff for the significant security benefits.
My optimization approach includes:
- Workload-specific configuration generated via PGTune (https://pgtune.leopard.in.ua/)
- Real-time performance monitoring with PgHero for identifying bottlenecks
- Automated VACUUM ANALYZE operations scheduled via pgcron targeting write-heavy tables, which prevents performance degradation and helps me sleep soundly
- A custom CLI utility I built for ZSTD-compressed backups that achieves impressive compression ratios while maintaining high throughput, with automatic S3 uploading: https://github.com/overflowy/pgbackup
This setup has been remarkably stable and performant, handling our workloads with substantial headroom for growth.
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#4Is this new async. I/O feature for Linux only? I know Windows has IOCP and also now an IORing implementation of its own (Less familiar with macOS capabilities other than POSIX AIO). https://learn.microsoft.com/en-us/windows/win32/api/ioringap... Update: Most of the comments below seem to be missing the fact that Windows now also has an IORing implementation, as I mentioned above. Comparison article here: https://wind…
Sounds like this feature is based on io_uring which is a Linux feature. I would be surprised if they implemented async io on Windows before they would on Linux given the user/deployment base being very Linux-heavy.
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#5On linux there also is preadv2(..., RWF_NOWAIT) which can be used to do optimistic non-blocking read from the page cache. That might be useful for io_method = worker to shave off a bit of latency. Try reading on the main thread with NOWAIT and only offload to a worker thread when that fails.
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#6Is this new async. I/O feature for Linux only? I know Windows has IOCP and also now an IORing implementation of its own (Less familiar with macOS capabilities other than POSIX AIO). https://learn.microsoft.com/en-us/windows/win32/api/ioringap... Update: Most of the comments below seem to be missing the fact that Windows now also has an IORing implementation, as I mentioned above. Comparison article here: https://wind…
[deleted]
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#7Is this new async. I/O feature for Linux only? I know Windows has IOCP and also now an IORing implementation of its own (Less familiar with macOS capabilities other than POSIX AIO). https://learn.microsoft.com/en-us/windows/win32/api/ioringap... Update: Most of the comments below seem to be missing the fact that Windows now also has an IORing implementation, as I mentioned above. Comparison article here: https://wind…
Sounds like this feature is based on io_uring which is a Linux feature. I would be surprised if they implemented async io on Windows before they would on Linux given the user/deployment base being very Linux-heavy.
For a long time ago there have been APIs to do asynchronous file I/O on the books for Linux but they weren't worth using because they didn't really speed anything up.
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#8I recently deployed Postgres on a dedicated Hetzner EX-44 server (20 cores, 64GB RAM, 2x 512GB NVMe SSDs in RAID 1) for €39/month. The price-to-performance ratio is exceptional, providing enterprise-level capacity at a fraction of typical cloud costs. For security, I implemented TailScale which adds only ~5ms of latency while completely eliminating public network exposure - a worthwhile tradeoff for the significant s…
That’s great, but you need a solid HA/backup and recovery strategy if you even remotely care about your data.
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#9Does anyone know when the update allowing more concurrent connections is dropping, so we can stop using pgbouncer?
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#10Is this new async. I/O feature for Linux only? I know Windows has IOCP and also now an IORing implementation of its own (Less familiar with macOS capabilities other than POSIX AIO). https://learn.microsoft.com/en-us/windows/win32/api/ioringap... Update: Most of the comments below seem to be missing the fact that Windows now also has an IORing implementation, as I mentioned above. Comparison article here: https://wind…
It depends on the I/O method - as described in the article, "io_uring" is only available on Linux (and requires building with liburing, as well as io_uring to be enabled in the Kernel), but the default (as of beta1) is actually "worker", which works on any operating system.
The "worker" method uses a dedicated pool of I/O worker processes that run in the background, and whilst not as performant as io_uring in our benchmark, did clearly outperform the "sync" method (which is the same as what Postgres currently has in 17 and older).