321–330 of 377 posts
Earlier quoted context omitted.
How does that make him immune to having dumb ideas? See, I'm judging the idea on merit. You're just defending your hero who has gone rogue.
I must have missed all the nuanced judgment in your original post. Maybe you can quote some for me.
Earlier quoted context omitted.
I think this is a great article that takes a maximalist point and that’s its flaw. You should rewrite code only when the cost of adding a new feature (one that is actually necessary) to the old codebase becomes comparable to designing your entire system from scratch to allow for that feature to be added easily. That is to say that the cost of the rewrite should become comparable to the cost of continuing development.…
I think it’s very dependent on how you use words like “rewrite” or “refactor”. The point the author makes about the two page function, and all the bug-fixes (lessons learned) makes sense only if you “rewrite” from scratch without looking at the history. You can absolutely “rewrite” the function in a manner that is “refactoring”, but will often get called “rewrite” in the real world. This may be because “refactor” is…
Earlier quoted context omitted.
Process isolation affects so many things in C. The strategy change is going to require changes to so many modules that it will either be a re-write or buggy. In practical terms, if every line needs to be audited and updated, it is a re-write
What makes you think that it will require that many changes? There will be some widespread mechanical changes (which can be verified to be complete with a bit of low level work, like a script using objdump/nm to look for non-TLS mutable variables) and some areas changing more heavily (e.g. connection establishment, crash detection, signal handling, minor details of the locking code). But large portions of the code wo…
Experience with other systems has taught me that in a system that's been in active use and development for decades, entanglement will be deep, subtle, and pervasive. If this isn't true of postgres then it's an absolute freak anomaly of a codebase. It is that in other ways, so it's possible.
But the article mentions there being thousands of global variables. And Tom Lane himself says he considers it untenable for exactly this reason. That's a very good reason to think that it will require that many changes imo.
For the record, I think this will be a disaster. There is far too much code that will get broken, largely silently, and much of it is not under our control. regards, tom lane (via https://lwn.net/ml/pgsql-hackers/4178104.1685978307@sss.pgh.... ) If Tom Lane says it will be a disaster, I believe it will be a disaster.
I don't expect you or others to buy into any particular code change at this point, or to contribute time into it. Just to accept that it's a worthwhile goal. If the implementation turns out to be a disaster, then it won't be accepted, of course. But I'm optimistic. The reply is much more reasonable than this blanket assertion of a disaster.
I'm not trying to naysay it per se, bc again I don't have technical knowledge of this codebase. But that's exactly the sort of scenario that can cause a large project to splinter or stall for years. Talking about "the implementation" absent the context that would be necessary to create that implementation seems naively optimistic, or at worst irresponsible.
I'm honestly surprised it took them so long to reach this conclusion. > That idea quickly loses its appeal, though, when one considers trying to create and maintain a 2,000-member structure, so the project is unlikely to go this way. As repulsive as this might sound at first, I've seen structures of hundreds of fields work fine if the hierarchy inside them is well organized and they're not just flat. Still, I have no…
This is how I made my fork of libtcc lock-free. Mainline has a lock so that all backends can use global variables, but only one instance can do codegen at a time. It was a giant refactoring Especially fun was when multiple compilation units used the same static variable name, but it all worked in the end.
Having been using and administering a lot of PostgreSQL servers, I hope they don't lose any stability over this. I've seen (and reported) bugs that caused panics/segfaults in specific psql processes. Not just connections, also processes related to wal writing or replication. The way it's built right now, a child process can be just forced to quit and it does not affect other processes. Hopefully switching into thread…
As a longstanding PG dev/DBA who doesn't know much about its internals, I would say that they should just move connection pooling into the main product.
Essentially, pgbouncer should be part of PG and should be able to manage connections with knowledge of what each connections is doing. That, plus, some sort of dynamic max connection setting based on what's actually going on.
That'll remove almost all the dev/DBA pain from separate processes.
Earlier quoted context omitted.
Of course it will. That's better than continue working with damaged memory structures and unpredictable consequences. For database it's more important than ever. Imagine writing corrupted data because other thread went crazy.
You're implying that only an OS can provide memory separation between units of execution - at least in .NET AppDomains give you the same protection within a single process, so why couldn't postgres have its own such mechanism? I'd also think with a database engine shared state is not just in-memory - i.e. one process can potentially corrupt the behaviour of another by what it writes to disk, so moving to a single-pro…
For native code, there's no such safety net. Likewise, even for managed language, an error in the interpreter code will still crash the VM, since there's nothing to fallback to anymore.
Earlier quoted context omitted.
Reminds me of PHP 6... For those who don't follow PHP closely - that version was an attempted refactor of the string implementation which essentially shut down nearly all work on PHP for a decade, stagnating the language until it became pretty terrible compared to other options. They finally gave up and started work on PHP 7 which uses the (perfectly good) PHP 5 strings. Ten years of wasted time by the best internal…
Things You Should Never Do https://www.joelonsoftware.com/2000/04/06/things-you-should-... An oldie but a goodie
Earlier quoted context omitted.
This is how I made my fork of libtcc lock-free. Mainline has a lock so that all backends can use global variables, but only one instance can do codegen at a time. It was a giant refactoring Especially fun was when multiple compilation units used the same static variable name, but it all worked in the end.
Out of curiosity, where is this fork? Sounds very interesting.
This is the multi-threaded compiler: https://github.com/rsaxvc/tcc-swarm
With the multi-threaded tcc above it scales about as well as multiprocess. With mainline it doesn't scale well at all.
So far I haven't gotten around to reusing anything across libtcc handles/instances, but would eventually like to share mmap()'d headers across instances, as well as cache include paths, and take invocation arguments through stdin one compilation unit per line.