Not sure what's going on here, but one connection per process seems... ancient.
Using threaded model is difficult, how about pre-fork? Some connections per process is a good improvement.
PostgreSQL reconsiders its process-based model
251–260 of 377 posts
Re: PostgreSQL reconsiders its process-based model
#252Unpopular idea: age limit on votes/contributions.
We are in a unique time period and generation that has strong opinions based on a history that only exists within itself.
At some point it needs to rewrite.
Re: PostgreSQL reconsiders its process-based model
#253Earlier quoted context omitted.
That'll likely be an even bigger task, and harder to get into mainline kernel. Linux multi-process is already pretty efficient compared to Windows. However, multi-process is inherently less efficient than multi-thread due to more safety predicates / isolation guaranteed by the kernel, I feel lowering it might lead to more security issues, similar to how Hyper Threading triggered a bunch of issues with Intel Processor…
Right - yeah I was really just wondering if some of the safety predicates could be reduced when there is a relationship between processes, such as the mitigations against cache attacks. I think the cache misses caused by multi-process were one of the reasons given that it's slower than threading. But I don't understand why this is necessarily the case given that the shared memory and executable text ultimately refer…
The TLB is basically keyed by (address space, virtual address % granularity), or needs to be flushed entirely when switching between different views of the address space (e.g. switching between processes). Unless your address space is exactly the same, you're at least going to duplicate TLB contents. Leading to a lower hit rate.
This isn't really an OS issue, more a hardware one, although potential hardware improvements would likely have to be explicitly utilized by operating systems.
Note that the TLB issue is different from the data / instruction cache situation.
Re: PostgreSQL reconsiders its process-based model
#254I wonder if it would be easier to create a C virtual machine that emulates all the OS interaction, then recompile Postgres and the extensions to run on this. Perhaps TruffleC would work?
Re: PostgreSQL reconsiders its process-based model
#255Earlier quoted context omitted.
From what I gather postgres isn't doing conventional IPC but instead it uses shared memory, which means the same mechanism threads use but with way higher complexity
As does Oracle, and others. I'm aware. IPC, to me, includes the conventional shared memory resources (memory segments, locks, semaphores, condition variable, etc.) used by these systems: resources acquired by processes for the purpose of communication with other processes. I get it though. The most general concept of shared memory is not coupled to an OS "process." You made me question whether my concept of term IPC…
Without a credible source to reconfirm what ChatGPT said, one can’t really assume what ChatGPT says is correct.
Re: PostgreSQL reconsiders its process-based model
#256A close to impossible task, if anyone can do it's probably Heikki though.
Unfortunately I expect this to go the way of zheap et al. Fundamental design changes like this have just had such a rough time of succeeding thus far.
I think for such a change to work it probably needs not just the support of Neon but also of say Microsoft (current stewards of Citus) that have larger engineering resources to throw at the problem and grind out all the bugs.
Re: PostgreSQL reconsiders its process-based model
#257Re: PostgreSQL reconsiders its process-based model
#258A close to impossible task, if anyone can do it's probably Heikki though. Unfortunately I expect this to go the way of zheap et al. Fundamental design changes like this have just had such a rough time of succeeding thus far. I think for such a change to work it probably needs not just the support of Neon but also of say Microsoft (current stewards of Citus) that have larger engineering resources to throw at the probl…
I know that at least people from EDB (Robert) and Microsoft (Thomas, me) are quite interested in eventually making this transition, it's not just Heikki. Personally I won't have a lot of cycles for the next release or two, but after that...
Re: PostgreSQL reconsiders its process-based model
#259Earlier quoted context omitted.
Reminds me of PHP 6... For those who don't follow PHP closely - that version was an attempted refactor of the string implementation which essentially shut down nearly all work on PHP for a decade, stagnating the language until it became pretty terrible compared to other options. They finally gave up and started work on PHP 7 which uses the (perfectly good) PHP 5 strings. Ten years of wasted time by the best internal…
Things You Should Never Do https://www.joelonsoftware.com/2000/04/06/things-you-should-... An oldie but a goodie
I think this is a great article that takes a maximalist point and that’s its flaw.
You should rewrite code only when the cost of adding a new feature (one that is actually necessary) to the old codebase becomes comparable to designing your entire system from scratch to allow for that feature to be added easily. That is to say that the cost of the rewrite should become comparable to the cost of continuing development. I have been a part of a couple of rewrites like that, one of them quite complex, and yes they were warranted and yes they worked.
But having said that you should absolutely be conservative with rewriting code. It’s a bad habit to always jump to a rewrite.
Re: PostgreSQL reconsiders its process-based model
#260Earlier quoted context omitted.
Reminds me of PHP 6... For those who don't follow PHP closely - that version was an attempted refactor of the string implementation which essentially shut down nearly all work on PHP for a decade, stagnating the language until it became pretty terrible compared to other options. They finally gave up and started work on PHP 7 which uses the (perfectly good) PHP 5 strings. Ten years of wasted time by the best internal…
Things You Should Never Do https://www.joelonsoftware.com/2000/04/06/things-you-should-... An oldie but a goodie
The issue here isn't "rewriting" per se but "stopping development".
You shouldn't stop development on your important products.
Letting a couple of your talented programmers loose on a greenfield reimplementation is a perfectly sane strategic move.
Stopping development on important products because you are 100% certain that the reimplementation will be successful by $DEADLINE is a foolish gamble.