[deleted]
PostgreSQL reconsiders its process-based model
11–20 of 377 posts
Re: PostgreSQL reconsiders its process-based model
#12That sounds like really hard programming. I’m glad I write react and get paid possibly much more.
I feel this sort of undertaking could only be done by those programmers who truly value domain knowledge above all else (money, etc). I'm more of the entrepreneureal mind so I generally only learn as much as needed to do some task (even if it's very difficult), but just seeking information as a means to an end doesn't feel fulfilling to me. Of course many people DO find that, and its upon those people's shoulders that heroic things like this rest, and I'm very thankful to them.
Re: PostgreSQL reconsiders its process-based model
#13I'm honestly surprised it took them so long to reach this conclusion. > That idea quickly loses its appeal, though, when one considers trying to create and maintain a 2,000-member structure, so the project is unlikely to go this way. As repulsive as this might sound at first, I've seen structures of hundreds of fields work fine if the hierarchy inside them is well organized and they're not just flat. Still, I have no…
Also, even if a 2k-member structure is obnoxious, consider the alternative - having to think about and manage 2k global variables is probably even worse!
Re: PostgreSQL reconsiders its process-based model
#14I hope they don't do it.
I've had a similar situation with PHP, where we had written quite a large engine (https://github.com/Qbix/Platform) with many features (https://qbix.com/features.pdf) . It took advantage of the fact that PHP isolated each script and gave it its own global variables, etc. In fact, much of the request handling did stuff like this:
Q_Request::requireFields(['a', 'b', 'c']);
$uri = Q_Dispatcher::uri();
$this->getContext()->request()->requireFields(['a', 'b', 'c']);
$this->getContext()->dispatcher()->uri();
amp
reactPHP
Swoole (native extension)
Fibers (inside PHP itself)
Then I realized... we would have to go through the entire code and redo how it all works. We'd also no longer benefit from PHP's process isolation. If one process crapped out or had a memory leak, it could take down everything else.
There's a reason PHP still runs 80% of all web servers in the world (https://kinsta.com/blog/is-php-dead/) ... and one of the biggest is that commodity servers can host terrible PHP code and it's mostly isolated in little processes that finish "quickly" before they can wreak havoc on other processes or on long-running stuff.
So now back to postgres. It's been praised for its rock-solid reliability and security. It's got so many features and the MVCC is very flexible. It seems to use a lot of global variables. They can spend their time on many other things, like making it byzantine-fault-tolerant, or something.
The clincher for me was when I learned that php-fpm (which spins up processes which sleep when waiting for I/O) is only 50% slower than all those fancy things above. Sure, PHP with Swoole can outperform even Node.js, and can handle twice as many requests. But we'd rather focus on soo many other things we need to do :)
Re: PostgreSQL reconsiders its process-based model
#15This feels like developers are bored and want a challenge.
Re: PostgreSQL reconsiders its process-based model
#16[deleted]
Re: PostgreSQL reconsiders its process-based model
#17This sounds like a problem that would border on the complexity of replacing the GIL in Ruby or Python. The performance benefits are obvious but it seems like the correctness problems would be myriad and a constant source of (unpleasant) surprises.
The correctness problem should be handled by a suite of automated tests which PostgreSQL has. If all tests pass, the application must work correctly. The project is too big, and has too many developers to make much progress without full test coverage. Where else would up-to-date documentation regarding the correct behavior of PostgreSQL exist? In some developers head? SQLite is pretty famous for there extreme approach to testing including out of memory conditions, and other rare circumstances: https://www.sqlite.org/testing.html
Re: PostgreSQL reconsiders its process-based model
#18Is there any reason at all people use intrinsically bug-prone and broken multithreading mode instead of fork() and IPC apart from WinAPI having no proper fork?
Re: PostgreSQL reconsiders its process-based model
#19Worked on a codebase which was separate processes, each of which has a shedload of global variables. It was a nightmare working out what was going on, not helped by the fact that there was no naming convention for the globals, plus they were not declared in a single place. I believe their use was a performance move, ie having the linker pin a var to a specific memory location rather than copying it to the stack as a variable and referencing it by offset the whole time. Premature optimisation? Optimisation at all? Who knows, but there's a good reason coding standards typically militate against globals.
Re: PostgreSQL reconsiders its process-based model
#20I recently looked through the source code of postgresql and every source files starts with a (really good) description of what the file is supposed to do, which made it really easy to get in to the code compared to other open source projects I've seen. So thanks for that.