Live data from Hacker News

PostgreSQL reconsiders its process-based model

lwn.net

261–270 of 377 posts

Re: PostgreSQL reconsiders its process-based model

#261

I wonder if it would be easier to create a C virtual machine that emulates all the OS interaction, then recompile Postgres and the extensions to run on this. Perhaps TruffleC would work? https://dl.acm.org/doi/10.1145/2647508.2647528

Hard to believe that would provide any benefit without also causing massive slowdowns.

Re: PostgreSQL reconsiders its process-based model

#262
post #251

Not sure what's going on here, but one connection per process seems... ancient. Using threaded model is difficult, how about pre-fork? Some connections per process is a good improvement.

The issue is costlier (runtime, complexity, memory) resource sharing, not the cost of the fork itself. Pre-forking isn't going to help with any of that.

Re: PostgreSQL reconsiders its process-based model

#263

Earlier quoted context omitted.

Reminds me of PHP 6... For those who don't follow PHP closely - that version was an attempted refactor of the string implementation which essentially shut down nearly all work on PHP for a decade, stagnating the language until it became pretty terrible compared to other options. They finally gave up and started work on PHP 7 which uses the (perfectly good) PHP 5 strings. Ten years of wasted time by the best internal…

Things You Should Never Do https://www.joelonsoftware.com/2000/04/06/things-you-should-... An oldie but a goodie

> Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make:

> They decided to rewrite the code from scratch.

Absolutely not what's being proposed for Postgres.

Re: PostgreSQL reconsiders its process-based model

#265

Sorry if I offend anybody, but this sounds like such a bad idea. I have been running various versions of postgres in production for 15 years with thousands of processes on super beefy machines, and I can tell you without a doubt that sometimes those processes crash - specially if you are running any of the extensions. Nevertheless, Postgres has 99% of the time proven to be resilient. The idea that a bad client can br…

Reading your comment makes me think it is not only a good idea, it is a necessity.

Relying on crashing as a bug recovery system is a good idea? Crashing is just part of the workflow? That's insane, and a good argument against PostgreSQL in any production system.

It is possible PostgreSQL doesn't migrate to a thread based model, and I am not arguing they should.

But debug and patch the causes of these crashes? Absolutely yes, and the sooner, the better.

Re: PostgreSQL reconsiders its process-based model

#266
post #256

A close to impossible task, if anyone can do it's probably Heikki though. Unfortunately I expect this to go the way of zheap et al. Fundamental design changes like this have just had such a rough time of succeeding thus far. I think for such a change to work it probably needs not just the support of Neon but also of say Microsoft (current stewards of Citus) that have larger engineering resources to throw at the probl…

Hey I'm fairly new to the who's who in the PostgreSQL world, would you mind telling why Heikki might be able to pull this off?

Re: PostgreSQL reconsiders its process-based model

#267

Earlier quoted context omitted.

Things You Should Never Do https://www.joelonsoftware.com/2000/04/06/things-you-should-... An oldie but a goodie

> Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make: > They decided to rewrite the code from scratch. Absolutely not what's being proposed for Postgres.

Process isolation affects so many things in C. The strategy change is going to require changes to so many modules that it will either be a re-write or buggy.

In practical terms, if every line needs to be audited and updated, it is a re-write

Re: PostgreSQL reconsiders its process-based model

#268
post #265

Sorry if I offend anybody, but this sounds like such a bad idea. I have been running various versions of postgres in production for 15 years with thousands of processes on super beefy machines, and I can tell you without a doubt that sometimes those processes crash - specially if you are running any of the extensions. Nevertheless, Postgres has 99% of the time proven to be resilient. The idea that a bad client can br…

Reading your comment makes me think it is not only a good idea, it is a necessity. Relying on crashing as a bug recovery system is a good idea? Crashing is just part of the workflow? That's insane, and a good argument against PostgreSQL in any production system. It is possible PostgreSQL doesn't migrate to a thread based model, and I am not arguing they should. But debug and patch the causes of these crashes? Absolut…

> Relying on crashing as a bug recovery system is a good idea? Crashing is just part of the workflow? That's insane

Erlang users don't seem to agree with you

Re: PostgreSQL reconsiders its process-based model

#269
post #158

Earlier quoted context omitted.

I have the opposite experience. Outdated comments are often way worse than no comments, because they can give you wrong ideas that aren't true anymore, and send you off in the wrong direction before you finally figure out the comment was wrong.

Indeed. I recently found this piece of code: if (X) assert(false); // we never do X, ever, anywhere. Then I look over to the other pane, where I have a different, but related file open: if (exact same X) { do_useful_stuff(); } It got a chuckle out of me.

Did you update the comment? :-)

Re: PostgreSQL reconsiders its process-based model

#270
post #230

Earlier quoted context omitted.

At most they'd be determined to be read only constants that are inlined during constant folding. This includes most integral sized / typed scalar values that fit into registers for the most part, and nothing you've taken the address of either - those remain as static data.

I think there might be a terminology mix-up here. In C, a global variable with the `static` keyword is is still mutable. So it typically can't be constant-folded/inlined. The `static` modifier in that context just means that the symbol is not exported, so other ".c" files can't access it.

A static variable in C is mutable in the same sense that a local variable is, but since it's not visible outside the current compilation unit the optimizer is allowed to observe that it's never actually modified or published and constant fold it away.

Check out the generated assembly for this simple program, notice that kBase is folded even though it's not marked const: https://godbolt.org/z/h45vYo5x5

Post reply on HN