Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL reconsiders its process-based model

lwn.net

Post reply on HN
Print view View original

191–200 of 377 posts

Re: PostgreSQL reconsiders its process-based model

#191

Post by wmf »

bb88 wrote: post #134

This reminds me of this poster: "You must be this tall..." https://bholley.net/blog/2015/must-be-this-tall-to-write-mul... Back about a decade ago I was "auditing" someone else's threaded code. And couldn't figure it out. But he was the company's "golden child" so by default it must be working code because he wrote it. And then it started causing deadlocks in prod. "What do you want me to do about it? It's the golden…

The thing is... multi-process with a bespoke shared memory system isn't better than multithreading; it's much worse.

Re: PostgreSQL reconsiders its process-based model

#192

Post by dfox »

cogman10 wrote: post #173
niccl wrote: post #139

Earlier quoted context omitted.

yes, but postmaster is still running to roll back the transaction. If you crash a single multi-threaded process, you may lose postmaster as well and then sadness would ensue

Transaction roll back is a part of the WAL. Databases write to the disk an intent to change things, what should be changed, and a "commit" of the change when finished so that all changes happen as a unit. If the DB process is interrupted during that log write then all changes associated with that transaction are rolled back. Threaded vs process won't affect that.

Running the whole DBMS as a bunch of threads in single process changes how fast is the recovery from some kind of temporary inconsistency. In the ideal world, this should not happen, but in reality it does and you do not want to bring the whole thing down because of some superficial data corruption.

On the other hand, all cases of fixable corrupted data in PostgreSQL I have seen were result of somebody doing something totally dumb (rsyncing live cluster, even between architectures), while on InnoDB it seems to happen somewhat randomly without any obvious reason of somebody doing stupid things.

Re: PostgreSQL reconsiders its process-based model

#193

Post by magicalhippo »

loeg wrote: post #7
mihaic wrote: post #2

I'm honestly surprised it took them so long to reach this conclusion. > That idea quickly loses its appeal, though, when one considers trying to create and maintain a 2,000-member structure, so the project is unlikely to go this way. As repulsive as this might sound at first, I've seen structures of hundreds of fields work fine if the hierarchy inside them is well organized and they're not just flat. Still, I have no…

Yeah. I think as a straightforward, easily correct transition from 2000 globals, a giant structure isn't an awful idea. It's not like the globals were organized before! You're just making the ambient state (awful as it is) explicit.

We did this with a project I worked on. I came on after the code was mature.

While we didn't have 2000 globals, we did have a non-trivial amount, spread over about 300kLOC of C++.

We started by just stuffing them into a "context" struct, and every function that accessed a global thus needed to take a context instance as a new parameter. This was tedious but easy.

However the upside was that this highlighted poor architecture. Over time we refactored those bits and the main context struct shrunk significantly.

The result was better and more modular code, and overall well worth the effort in our case, in my opinion.

Re: PostgreSQL reconsiders its process-based model

#194

Post by mike_hearn »

mastax wrote: post #138

It would be interesting to have something between threads and processes. I'll call them heavy-threads for sake of discussion. Like light-threads, heavy-threads would share the same process-security-boundary and therefore switching between them would be cheap. No need to flush TLB, I$, D$. Like processes, heavy-threads would have mostly-separate address spaces by default. Similar to forking a process, they could share…

There are some problems.

1. Mostly separate address spaces requires changing the TLB on context switch (modern hw lets it be partial). You could use MPKs to share a single address space with fast protection switches.

2. Threads share the global heap, but your heavy threads would require explicitly defined shared writeable memory regions, so presumably each one has its own heap. That's a fair bit of overhead.

3. Failure isolation is more complicated than deciding what to kill.

The expand on the last point, Postgres doesn't isolate failures to a single process because they do share memory and might corrupt those shared memory regions. But even if you don't have shared memory failure recovery isn't always easy. Software has to be written specifically to plan for it. You can kill processes because everything in the OS is written around allowing for that possibility, for example, shells know what to do if a sub-process is killed unexpectedly. Killing a heavy thread (=process) is no good if the parent process is going to wait for a reply from it forever because it wasn't written to handle the process going away.

Re: PostgreSQL reconsiders its process-based model

#196

Post by dfox »

dan-robertson wrote: post #174
rburhum wrote: post #113

Sorry if I offend anybody, but this sounds like such a bad idea. I have been running various versions of postgres in production for 15 years with thousands of processes on super beefy machines, and I can tell you without a doubt that sometimes those processes crash - specially if you are running any of the extensions. Nevertheless, Postgres has 99% of the time proven to be resilient. The idea that a bad client can br…

Is the actual number you got 99%? Seems low to me but I don’t really know about Postgres. That’s 3 and a half days of downtime per year, or an hour and a half per week.

Well, hour and half per week is the amount of downtime that you need for modestly sized database (units of TB) accessed by legacy clients that have ridiculously long running transactions that interfere with autovacuum.

Re: PostgreSQL reconsiders its process-based model

#197

Post by vbezhenar »

wizofaus wrote: post #141
vbezhenar wrote: post #137

Earlier quoted context omitted.

Of course it will. That's better than continue working with damaged memory structures and unpredictable consequences. For database it's more important than ever. Imagine writing corrupted data because other thread went crazy.

You're implying that only an OS can provide memory separation between units of execution - at least in .NET AppDomains give you the same protection within a single process, so why couldn't postgres have its own such mechanism? I'd also think with a database engine shared state is not just in-memory - i.e. one process can potentially corrupt the behaviour of another by what it writes to disk, so moving to a single-pro…

I don't know .NET enough to comment here, but I'm pretty sure that if you would manage to run bare metal C inside your .NET app (should be possible), it'll destroy all your domains easily. RAM is RAM. The only memory protection that we have is across process boundary (even that protection is not perfect with shared memory, but at least it allows to protect private memory).

At least I'm not aware of any way to protect private thread memory from other threads.

Postgres is C and that's not going to change ever.

Re: PostgreSQL reconsiders its process-based model

#199

Post by dfox »

cogman10 wrote: post #176
loeg wrote: post #7

Earlier quoted context omitted.

Yeah. I think as a straightforward, easily correct transition from 2000 globals, a giant structure isn't an awful idea. It's not like the globals were organized before! You're just making the ambient state (awful as it is) explicit.

I think my bigger fear is around security. A process per connection keeps things pretty secure for that connection regardless of what the global variables are doing (somewhat hard to mess that up with no concurrency going on in a process). Merge all that into one process with many threads and it becomes a nightmare problem to ensure some random addon didn't decide to change a global var mid processing (which causes w…

All postgres processes run under the same system user and all the access checking happens completely in userspace.

Re: PostgreSQL reconsiders its process-based model

#200

Post by wizofaus »

vbezhenar wrote: post #197
wizofaus wrote: post #141

Earlier quoted context omitted.

You're implying that only an OS can provide memory separation between units of execution - at least in .NET AppDomains give you the same protection within a single process, so why couldn't postgres have its own such mechanism? I'd also think with a database engine shared state is not just in-memory - i.e. one process can potentially corrupt the behaviour of another by what it writes to disk, so moving to a single-pro…

I don't know .NET enough to comment here, but I'm pretty sure that if you would manage to run bare metal C inside your .NET app (should be possible), it'll destroy all your domains easily. RAM is RAM. The only memory protection that we have is across process boundary (even that protection is not perfect with shared memory, but at least it allows to protect private memory). At least I'm not aware of any way to protect…

I certainly wasn't suggesting it would make sense to rewrite Postgres to run on .NET (using any language, even managed C++, assuming anyone still uses that). Yes, it's inherent in the C/C++ language that it's able to randomly access any memory that a process has access to, and obviously on that basis OS-provided process-separation is the "best" protection you can get, just pointing out that it's not the only possibility.
Post reply on HN