Is io_uring still plagued by security issues enabled by it's use? Or have those largely been fixed? My understanding was many Linux admins (or even distros by default?) were disabling io_uring.
Disabling io_uring because “guy on the internet said so” or “$faang_company says so” is beyond dumb. One should evaluate the risk according to their specific use case. It can be a good idea to disable it of you run untrusted workloads (eg: other people’s containers, sharing the same kernel) but if you have a kernel on a machine (virtual or real) dedicated to your own workload you can pretty much keep using io_uring.…
Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
121–130 of 159 posts
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#122Postgres is such a cool project, I have so much respect for its maintainers and community! For me its the second most impactful OSS project in the business tech world behind Linux itself. A real public good to be cherished and praised.
I recently started a ML project using text data and the choice was between MySQL and Postgres. Having looked at the respective features and pros and cons, the choice was immediately obvious. Also, with pgvector and https://postgresml.com available, the choice for Postgres was even easier.
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#123Is this new async. I/O feature for Linux only? I know Windows has IOCP and also now an IORing implementation of its own (Less familiar with macOS capabilities other than POSIX AIO). https://learn.microsoft.com/en-us/windows/win32/api/ioringap... Update: Most of the comments below seem to be missing the fact that Windows now also has an IORing implementation, as I mentioned above. Comparison article here: https://wind…
I am not a Windows guy but I (with help) managed to get IOCP working for this in a basic prototype. Will share publicly soon. I also sketched out an IoRing version (if you are interested in helping debug and flesh that out let me know!). Main learnings: the IOCP version can't do asynchronous flush! Which we want. The IoRing version can! But it can't do scatter/gather AKA vector I/O yet! Which is an essential feature…
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#124Earlier quoted context omitted.
Yes, although Windows has had async I/O since Windows NT 3.1, their API is still not supported by Postgres.
FWIW, there are prototype patches for an IOCP based io_method. We just couldn't get them into an acceptable state for PG 18. I barely survived getting in what we did...
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#125Postgres is such a cool project, I have so much respect for its maintainers and community! For me its the second most impactful OSS project in the business tech world behind Linux itself. A real public good to be cherished and praised.
I recently started a ML project using text data and the choice was between MySQL and Postgres. Having looked at the respective features and pros and cons, the choice was immediately obvious. Also, with pgvector and https://postgresml.com available, the choice for Postgres was even easier.
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#126Does anyone know when the update allowing more concurrent connections is dropping, so we can stop using pgbouncer?
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#127Earlier quoted context omitted.
I recently started a ML project using text data and the choice was between MySQL and Postgres. Having looked at the respective features and pros and cons, the choice was immediately obvious. Also, with pgvector and https://postgresml.com available, the choice for Postgres was even easier.
Why not MongoDB?
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#128Earlier quoted context omitted.
I recently started a ML project using text data and the choice was between MySQL and Postgres. Having looked at the respective features and pros and cons, the choice was immediately obvious. Also, with pgvector and https://postgresml.com available, the choice for Postgres was even easier.
Why not MongoDB?
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#129This looks promising! Wonder if it's coming to Neon
And we plan to have 18 out as quickly as we did 17; on the day of release.
Re: Waiting for Postgres 18: Accelerating Disk Reads with Asynchronous I/O
#130Postgres is such a cool project, I have so much respect for its maintainers and community! For me its the second most impactful OSS project in the business tech world behind Linux itself. A real public good to be cherished and praised.