Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL reconsiders its process-based model

lwn.net

Post reply on HN
Print view View original

281–290 of 377 posts

Re: PostgreSQL reconsiders its process-based model

#281

Post by pphysch »

paulddraper wrote: post #271
pphysch wrote: post #260

Earlier quoted context omitted.

The issue here isn't "rewriting" per se but "stopping development". You shouldn't stop development on your important products. Letting a couple of your talented programmers loose on a greenfield reimplementation is a perfectly sane strategic move. Stopping development on important products because you are 100% certain that the reimplementation will be successful by $DEADLINE is a foolish gamble.

The problem is if it snowballs

Any project, whether it's a "rewrite" or not, can succumb to scope creep and poor project management.

Re: PostgreSQL reconsiders its process-based model

#282

Post by devjab »

IgorPartola wrote: post #259
SoftTalker wrote: post #241

Earlier quoted context omitted.

Things You Should Never Do https://www.joelonsoftware.com/2000/04/06/things-you-should-... An oldie but a goodie

I think this is a great article that takes a maximalist point and that’s its flaw. You should rewrite code only when the cost of adding a new feature (one that is actually necessary) to the old codebase becomes comparable to designing your entire system from scratch to allow for that feature to be added easily. That is to say that the cost of the rewrite should become comparable to the cost of continuing development.…

I think it’s very dependent on how you use words like “rewrite” or “refactor”. The point the author makes about the two page function, and all the bug-fixes (lessons learned) makes sense only if you “rewrite” from scratch without looking at the history. You can absolutely “rewrite” the function in a manner that is “refactoring”, but will often get called “rewrite” in the real world. This may be because “refactor” is sort of this English CS term that doesn’t have a real translation or usage in many languages and “rewrite” is sort of universal for changing text, but in CS is sort of “rebuilding” things.

I don’t think you necessarily need to be conservative about rewriting things. We do it all the time in fact. We build something to get it out there and see the usage, and then we build it better and then we do it again. Which often involves a lot of “rewriting” but thanks to principles like SOLID’s single responsibility makes this rather easy to both do and maintain (we write a lot of semi-functional code and try to avoid using OOP unless necessary, so we don’t really use all the parts of SOLID religiously).

I do agree that it’s never a good idea to get into things with the mind-set of “we can do this better if we start from scratch” because you can’t.

Re: PostgreSQL reconsiders its process-based model

#283

Post by devjab »

pphysch wrote: post #260
SoftTalker wrote: post #241

Earlier quoted context omitted.

Things You Should Never Do https://www.joelonsoftware.com/2000/04/06/things-you-should-... An oldie but a goodie

The issue here isn't "rewriting" per se but "stopping development". You shouldn't stop development on your important products. Letting a couple of your talented programmers loose on a greenfield reimplementation is a perfectly sane strategic move. Stopping development on important products because you are 100% certain that the reimplementation will be successful by $DEADLINE is a foolish gamble.

Isn’t that how you end up with Python 2 and 3 though?

Re: PostgreSQL reconsiders its process-based model

#284

Post by jupp0r »

orthoxerox wrote: post #179
jupp0r wrote: post #73

Please don't use mutable global state in your work. Global variables are universally bad and don't provide much of a benefit. The number of desirable architectural refactoring that I've witnessed turning into a muddy mess because of them is daunting. This is one more example of this.

You know what a database is, do you? It is the place where you store your mutable global state. You can't kick the can down the road forever, someone has to tackle the complexity of managing state.

Databases are great, especially those who do not use global variables in their implementation.

Re: PostgreSQL reconsiders its process-based model

#285

Post by masklinn »

bsder wrote: post #148
masklinn wrote: post #32

Earlier quoted context omitted.

An other component of the Gil story is that removing the Gil require adding fine grained locks, which (aside from making VM development more complicated) significantly increases lock traffic and thus runtime costs, which noticeably impacts single-threaded performance, which is of major import. Postgres starts from a share-nothing architecture, it’s quite a bit easier to evaluate the addition of sharing.

> which noticeably impacts single-threaded performance, which is of major import. 1) I don't buy this a priori. Almost everybody who removed a gigantic lock suddenly realizes that there was more contention than they thought and that atomizing it made performance improve. 2) Had Python bitten the bullet and removed the GIL back at Python 3.0, the performance would likely already be back to normal or better. You can't…

> I don't buy this a priori.

You don’t have to buy anything, that’s been the result of every attempt so far and a big reason for their rejection. The latest effort only gained some traction because the backers also did optimisation work which compensated (and then was merged separately).

> Almost everybody who removed a gigantic lock

See that’s the issue with your response, you’re not actually reading the comment you’re replying to.

And the “almost” is a big tell.

> suddenly realizes that there was more contention than they thought and that atomizing it made performance improve.

There is no contention on the gil in single threaded workloads.

> Had Python bitten the bullet and removed the GIL back at Python 3.0

It would have taken several more years and been completely DOA.

Re: PostgreSQL reconsiders its process-based model

#286

Post by jupp0r »

slashdev wrote: post #78
jupp0r wrote: post #73

Please don't use mutable global state in your work. Global variables are universally bad and don't provide much of a benefit. The number of desirable architectural refactoring that I've witnessed turning into a muddy mess because of them is daunting. This is one more example of this.

Thank you for sharing your ideological views, but this is not the appropriate venue for that. If you want to have a software _engineering_ discussion about the trade offs involved in sharing global mutable state, this is a good venue for that. All engineering is trade offs. As soon as you make blanket statements that X is always bad, you’ve transitioned into the realm of ideology. Now presumably you mean to say it’s…

Discrediting my argument by labeling it as ideology and by implying that "blanket statements are always bad" is a logical fallacy that does not touch the merits of what is discussed and I would argue that your argument instead of mine is the one that does not belong here.

If you want to contribute to the discussion, I'd be happy to be given an example of successful usage of global variables that made a project a long term success under changing requirements compared to the alternatives.

Re: PostgreSQL reconsiders its process-based model

#287

Post by pmontra »

robomc wrote: post #237
abhibeckert wrote: post #224

Earlier quoted context omitted.

Reminds me of PHP 6... For those who don't follow PHP closely - that version was an attempted refactor of the string implementation which essentially shut down nearly all work on PHP for a decade, stagnating the language until it became pretty terrible compared to other options. They finally gave up and started work on PHP 7 which uses the (perfectly good) PHP 5 strings. Ten years of wasted time by the best internal…

I have the "Professional PHP6" book which I feel like should be a collectors item or something. Weird book IMO, because it has a lot of content that's just about general software development, rather than anything to do with PHP specifically, or the theoretical PHP6 APIs in particular.

PHP used to be the first computer language learned by people wanting to create a scripted web page. This was more true in the 90s but maybe it stuck. So it would be OK to add some general guidance about writing software and organizing projects.

Re: PostgreSQL reconsiders its process-based model

#288

Post by kccqzy »

eclipticplane wrote: post #72

For the record, I think this will be a disaster. There is far too much code that will get broken, largely silently, and much of it is not under our control. regards, tom lane (via https://lwn.net/ml/pgsql-hackers/4178104.1685978307@sss.pgh.... ) If Tom Lane says it will be a disaster, I believe it will be a disaster.

    I don't expect you or others to buy into any particular code change at 
    this point, or to contribute time into it. Just to accept that it's a 
    worthwhile goal. If the implementation turns out to be a disaster, then 
    it won't be accepted, of course. But I'm optimistic.
The reply is much more reasonable than this blanket assertion of a disaster.

Re: PostgreSQL reconsiders its process-based model

#289

Post by h0l0cube »

yxre wrote: post #267
h0l0cube wrote: post #263

Earlier quoted context omitted.

> Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make: > They decided to rewrite the code from scratch. Absolutely not what's being proposed for Postgres.

Process isolation affects so many things in C. The strategy change is going to require changes to so many modules that it will either be a re-write or buggy. In practical terms, if every line needs to be audited and updated, it is a re-write

> either be a re-write or buggy

A large refactor at best. It will touch lots of parts of the code base, but the vast majority of the source code would remain intact. Otherwise they could just Rewrite it in Rust™ while they’re at it

> if every line needs to be audited and updated, it is a re-write

I’m not sure why you believe every line needs to be updated. Most code is thread agnostic.

Re: PostgreSQL reconsiders its process-based model

#290

Post by jupp0r »

slashdev wrote: post #111
refulgentis wrote: post #81

Earlier quoted context omitted.

Global mutable state being a poor choice in software architecture isn’t an ideology. There is no ideology that argues it is awesome. If you want to have a software _engineering_ discussion about the trade offs involved in sharing global mutable state, this is a good venue for that. All engineering is trade offs. As soon as you start telling people they’re making blanket statements that X is always bad, you’ve transit…

It's awesome where performance considerations are paramount. It's awesome in databases. It's awesome in embedded software. It's awesome in operating system kernels. The fact is sometimes it's good. Saying it's universally bad is going beyond the realm of logic and evidence and into the realm of ideology.

Can you explain how having a global variable is more performant than passing a pointer to an object as a function argument in practice?
Post reply on HN