AWS engineer reports PostgreSQL perf halved by Linux 7.0, fix may not be easy
1–10 of 177 posts
Re: AWS engineer reports PostgreSQL perf halved by Linux 7.0, fix may not be easy
#2Its worth reading this follow-up LKML post by Andres Freund (who works on Postgres): https://lore.kernel.org/lkml/yr3inlzesdb45n6i6lpbimwr7b25kqk...
Re: AWS engineer reports PostgreSQL perf halved by Linux 7.0, fix may not be easy
#3Once upon a time, Linus would shout and yell about how the kernel should never "break" userspace (and I see in some places, some arguments of "It's not broken, it's just a performance regression" - personally I'd argue a 50% hit to performance of a pre-eminent database engine is ... quite the regression).
Now, the kernel engineer who introduced the brand new mechanism (introduced in Linux 7.0) for handling pre-emption says the "fix" is for Postgres to start using this new mechanism (I think the sister comment below links to what one of the Postgres engineers thinks of that, and I'm inclined mostly to agree).
Re: AWS engineer reports PostgreSQL perf halved by Linux 7.0, fix may not be easy
#4Nobody sensible runs the latest kernel; nobody running PG in production should be afraid of setting a non-default at either boot time or as a sysctl. So this will, most likely, be another step in building a PG database server (turn off pre-emption if your kernel is 7.0 or later and PG is pre-whatever-version).
At worst it might become a permanent part of building a PG server and a FAQ... but if it affects one thing this badly, it will affect others.
Re: AWS engineer reports PostgreSQL perf halved by Linux 7.0, fix may not be easy
#5It's not a good look to break userspace applications without a deprecation period where both old and new solutions exist, allowing for a transition period.
Re: AWS engineer reports PostgreSQL perf halved by Linux 7.0, fix may not be easy
#6Nobody sensible runs the latest kernel; nobody running PG in production should be afraid of setting a non-default at either boot time or as a sysctl. So this will, most likely, be another step in building a PG database server (turn off pre-emption if your kernel is 7.0 or later and PG is pre-whatever-version). At worst it might become a permanent part of building a PG server and a FAQ... but if it affects one thing t…
That may be the case, but it’s still not a great situation to be in and one has to wonder: if PostgreSQL is affected, what else is?
Re: AWS engineer reports PostgreSQL perf halved by Linux 7.0, fix may not be easy
#7Its worth reading this follow-up LKML post by Andres Freund (who works on Postgres): https://lore.kernel.org/lkml/yr3inlzesdb45n6i6lpbimwr7b25kqk...
Funny how "use hugepages" is right there on the table and 99% of users ignore it.
Re: AWS engineer reports PostgreSQL perf halved by Linux 7.0, fix may not be easy
#8Nobody sensible runs the latest kernel; nobody running PG in production should be afraid of setting a non-default at either boot time or as a sysctl. So this will, most likely, be another step in building a PG database server (turn off pre-emption if your kernel is 7.0 or later and PG is pre-whatever-version). At worst it might become a permanent part of building a PG server and a FAQ... but if it affects one thing t…
> Nobody sensible runs the latest kernel
From the article: "Linux 7.0 stable is due out in about two weeks. This is also the kernel version powering Ubuntu 26.04 LTS to be released later in April."
Unfortunately, lots of people will be running it in less than a month. At the moment, it'll take a kernel patch (not a sysctl) to undo this-- hopefully something changes.
Re: AWS engineer reports PostgreSQL perf halved by Linux 7.0, fix may not be easy
#9Once upon a time, Linus would shout and yell about how the kernel should never "break" userspace (and I see in some places, some arguments of "It's not broken, it's just a performance regression" - personally I'd argue a 50% hit to performance of a pre-eminent database engine is ... quite the regression). Now, the kernel engineer who introduced the brand new mechanism (introduced in Linux 7.0) for handling pre-emptio…
> I'd argue a 50% hit to performance [...] is ... quite the regression
Indeed! Especially if said regression happens to impact anything trade/market related...
Re: AWS engineer reports PostgreSQL perf halved by Linux 7.0, fix may not be easy
#10Background on PREEMPT_LAZY: