I am going to go ahead and trust Tom Lane on this one, over someone who is working on "serverless Postgres". Godspeed to the forthcoming fork.
PostgreSQL reconsiders its process-based model
71–80 of 377 posts
Re: PostgreSQL reconsiders its process-based model
#72 For the record, I think this will be a disaster. There is far too much
code that will get broken, largely silently, and much of it is not
under our control.
regards, tom lane
If Tom Lane says it will be a disaster, I believe it will be a disaster.
Re: PostgreSQL reconsiders its process-based model
#73Please don't use mutable global state in your work. Global variables are universally bad and don't provide much of a benefit. The number of desirable architectural refactoring that I've witnessed turning into a muddy mess because of them is daunting. This is one more example of this.
Re: PostgreSQL reconsiders its process-based model
#74ngmi
Re: PostgreSQL reconsiders its process-based model
#75Earlier quoted context omitted.
Exactly, if you're now forced to put everything in one place you're forced to acknowledge and understand the complexity of your state, and might have incentives to simplify it.
Here's MySQL's all-session-globals-in-one-place-class: https://github.com/mysql/mysql-server/blob/8.0/sql/sql_class... I believe I can safely say that nobody acknowledges and understands the complexity of all state within that class, and that whatever incentives there may be to simplify it are not enough for that to actually happen. (It ends on line 4692)
Right but that would still be true if they were globals instead. Putting all the globals in a class doesn't make any difference to how much state you have.
Re: PostgreSQL reconsiders its process-based model
#76I'm curious if they can take advantage of vfork / CLONE_VM, to get the benefits of sharing memory and lower overhead context switches, with the trade of still getting benefits from the scheduler, and sysadmin-friendliness. The other thing that might be interesting is FUTEX_SWAP / UMCG. Although it doesn't remove the overhead induced by context switches entirely (specifically, you would still deal with TLB misses), yo…
what about support for Windows?
Re: PostgreSQL reconsiders its process-based model
#77Earlier quoted context omitted.
I've never really been limited by CPU when running postgres (few TB instances). The bottleneck is always IO. Do others have different experience? Plus there's elegance and a feeling of being in control when you know query is associated with specific process which you can deal with and monitor just like any other process. But I'm very much clueless about internals, so this is a question rather than an opinion.
I see postgres become CPU bound regularly: Lots of hash joins, copy from or to CSV, index or materialized view rebuild. Postgis eats CPU. Tds_fdw tends to spend a lot of time doing charset conversion, more than actually networking to mssql. I was surprised when starting with postgres. Then again, I have smaller databases (A few TB) and the cache hit ratio tends to be about 95%. Combine that with SSDs, and it becomes…
I've also definitely seen a lot of CPU bounding on postgres.
Re: PostgreSQL reconsiders its process-based model
#78Please don't use mutable global state in your work. Global variables are universally bad and don't provide much of a benefit. The number of desirable architectural refactoring that I've witnessed turning into a muddy mess because of them is daunting. This is one more example of this.
Thank you for sharing your ideological views, but this is not the appropriate venue for that. If you want to have a software _engineering_ discussion about the trade offs involved in sharing global mutable state, this is a good venue for that. All engineering is trade offs. As soon as you make blanket statements that X is always bad, you’ve transitioned into the realm of ideology. Now presumably you mean to say it’s almost always bad. But that really depends on the context. It may well be almost always bad in average software projects, but PostgreSQL is not your average software project. Databases are a different realm.
Re: PostgreSQL reconsiders its process-based model
#79I recently looked through the source code of postgresql and every source files starts with a (really good) description of what the file is supposed to do, which made it really easy to get in to the code compared to other open source projects I've seen. So thanks for that.
I have no idea why that isn't standard practice in every codebase. I should be able to figure out your code without having to ask, or dig through issues or commit messages. Just tell me what it's for!
Because it takes a lot of time and because the comments can get outdated. I also want this for all my code bases. But do I always do this myself? No, especially on green field projects. I will sometimes go back and annotate them later.
Re: PostgreSQL reconsiders its process-based model
#80I'm honestly surprised it took them so long to reach this conclusion. > That idea quickly loses its appeal, though, when one considers trying to create and maintain a 2,000-member structure, so the project is unlikely to go this way. As repulsive as this might sound at first, I've seen structures of hundreds of fields work fine if the hierarchy inside them is well organized and they're not just flat. Still, I have no…
Yeah. I think as a straightforward, easily correct transition from 2000 globals, a giant structure isn't an awful idea. It's not like the globals were organized before! You're just making the ambient state (awful as it is) explicit.
Yes, it’s the most pragmatic and it’s only “awful” because it makes the actual problem visible. And would likely encourage slowly refactoring code to handle its state in a more sane way, until you’re only left with the really gnarly stuff, which shouldn’t be too much anymore and you can put them in individual thread local storages.
It’s an easy transition path.