The PostgreSQL documentation and the limitations of community
rhaas.blogspot.com
The PostgreSQL documentation and the limitations of community
1–10 of 61 posts
Re: The PostgreSQL documentation and the limitations of community
#2I think the main limitation of our docs is that it mostly explains what the pieces do, not how to use them to achieve a particular goal. For example, we have pretty good documentation of all the pieces to do HA, we just don't tell people how to assemble them together.
The reason is, I think, that flexibility is a pretty fundamental part of the project. We're great at providing building blocks (and documenting them), but we steer clear of describing a particular way to assemble them together.
For example, we might describe a particular HA approach, but then that would be perceived as "recommended / official" way, giving it preference over other (and equally valid) approaches and tooling. These "how to" docs are bound to be way more opinionated, so we just focus on documenting the pieces.
In other words, our docs are written by devs for devs, and we leave the higher level stuff to tutorials written by others etc.
Re: The PostgreSQL documentation and the limitations of community
#3As a regular consumer of pg docs, I vehemently agree that they are incredibly detailed, and at the same time, daunting to navigate.
Re: The PostgreSQL documentation and the limitations of community
#4This is a very valuable post in my opinion. Documentation is so critical to open source as well as to the private sector.
It can make the difference in how long it takes you to complete a project at work.
Re: The PostgreSQL documentation and the limitations of community
#5I think the main limitation of our docs is that it mostly explains what the pieces do, not how to use them to achieve a particular goal. For example, we have pretty good documentation of all the pieces to do HA, we just don't tell people how to assemble them together. The reason is, I think, that flexibility is a pretty fundamental part of the project. We're great at providing building blocks (and documenting them),…
This reminds me of technical documentation for embedded devices. Usually you get multiple classes of documents: data sheets, application notes, reference designs, user guides, erratas.
The problems described come from trying to be everything in one place, but it does not have to be. As I understand you try to be mostly a data sheet, which is probably a net good, because it is the document needed to be maintained, even if hard to navigate.
However, there are more document classes that can be produced. Yes, a reference design is inevitably going to be opinionated, whether it is produced by project team or some internet person. A reference design produced by project team at least has a fighting chance at staying somewhat up to date. And one can discuss tradeoffs between different approaches in an application note.
Re: The PostgreSQL documentation and the limitations of community
#6This is a very valuable post in my opinion. Documentation is so critical to open source as well as to the private sector. It can make the difference in how long it takes you to complete a project at work.
At this point I believe that documentation is the most important marketing artifact that a project has.
Re: The PostgreSQL documentation and the limitations of community
#7Here's my perspective. I've been a technical writer (TW) for ~10 years. 3 at an IoT startup, 7 at Google.
> The strengths of this process are also its weaknesses. A developer is, by definition, someone who spends the majority of their time doing development, which is to say writing code. Updating the documentation becomes a task that must be completed so that the code one has written can get committed so that one can move on to the next project and write some more code.
I may be misinterpreting, but I get the sense that the author feels that there is some kind of more optimal way to split up docs duties. IMO there is not. At least, not for reference docs. As the author said, the people implementing the code are in the best position to keep the reference information up-to-date.
If I grokked the rest of the article correctly, the author is essentially saying that the engineers have trouble writing and maintaining the other main types of docs [1] --- guides, tutorials, and overviews (explanations). It also sounds like they are having a "too many cooks in the kitchen" problem with pushing through changes to the other docs. I have a simple answer to that: hire some strong technical writers and make it clear to everyone that the TWs are Responsible and Accountable [2] for those docs. Also, make it explicit that the engineers are a Consulted role when it comes to guides / tutorials / overviews. Writing these types of docs is hard, specialized work. As the author said, the engineers have lots of other priorities. Of course, it's a bit self-serving for a TW to say "the solution is to hire TWs" but I get the sense that people don't realize that the easiest way to get good guides / tutorials / overviews is to hire people who have thought long and hard about those specialized tasks. If you want a good database, you don't expect your TWs to do the job. You get a database engineer. If you want good tutorials / guides / overviews, you likewise shouldn't expect your database engineer to do the job. You get TWs.
[2] https://en.m.wikipedia.org/wiki/Responsibility_assignment_ma...
Re: The PostgreSQL documentation and the limitations of community
#8I think the main limitation of our docs is that it mostly explains what the pieces do, not how to use them to achieve a particular goal. For example, we have pretty good documentation of all the pieces to do HA, we just don't tell people how to assemble them together. The reason is, I think, that flexibility is a pretty fundamental part of the project. We're great at providing building blocks (and documenting them),…
Another good example is the differences in the documentation for Indexes, vs the https://use-the-index-luke.com/ that explains many of the reasons WHY you want to organize it with great examples.
A problem I have is so many tutorials, or 'best practices' I find on the internet are for older versions that don't really apply as well in newer versions of postgres. Like searching for logical replication, you find lots of information for pg_logical for older versions of postgres, but many of those parts are now baked into postgres, but with a different syntax, etc.
I would love to see a 'tutorials/guide' and 'best practices' part of the documentation that is updated with each new release, that give examples of the most common tasks, and when/why to use them, and when to move to something more advanced.
Some really basic stuff like "this is the 3 best ways to handle replication in version 15, and the 2 or 3 most common ways to do backups, or these are the recommended ways to migrate from the previous version either in place, or to a new server, etc.
Re: The PostgreSQL documentation and the limitations of community
#9I love the approach of “change the code, write the documentation”; the code author is in a unique position to be able to explain the new behavior of the system. However, most FOSS projects could benefit from a technical writer, who can improve these first drafts to make them more usable.
Back in the Stone Age, companies like IBM had vast writing staffs. As a result, you got entire walls full of documentation. For OS/360, you got Concepts And Facilities, that explained what the software did for you; you got reference manuals; and Program Logic Manuals, which explained, in exhaustive/ing detail, how the program worked (with flowcharts!).
Arguably, the software I use today is far more complex than OS/360; yet we don't see the same attention to organization and detail in the documentation. I understand the reasons: IBM's dead-tree wall relied on paying tech writers, and that's incompatible with the tiny budgets most FOSS projects suffer. Far too often, I will go to a FOSS site's documentation page and discover a mass of links to pages that explain how to build the program on Itanium, or why this program is better than another one, with no walkthrough of “how to use this program for a typical case”, or even “if you want to do this kind of operation, these are the pages you need to read”. Often, the documentation isn't included in the repo, which lowers my confidence that the documentation and software are updated in step.
So I celebrate projects such as PostgreSQL, Emacs, and Arch Linux (for the wiki)—there are many more—where there is a real effort to create good documentation, even when I think that documentation can be improved or reorganized. Let's not allow the perfect to be the enemy of the good.
Re: The PostgreSQL documentation and the limitations of community
#10I think the main limitation of our docs is that it mostly explains what the pieces do, not how to use them to achieve a particular goal. For example, we have pretty good documentation of all the pieces to do HA, we just don't tell people how to assemble them together. The reason is, I think, that flexibility is a pretty fundamental part of the project. We're great at providing building blocks (and documenting them),…
These concerns seem to be specific to cases where there are various competing high-level design "strategies" with political weight behind them.
There are cases where PG is missing high-level docs where I don't think this applies.
For example, there's no official doc on how to write PL/pgSQL code. There's just an extremely-low-level language reference, covering each syntax element separately. There's no cookbook (other than the few examples per syntax element that exist to document the edge-cases of use of that syntax element); no tutorial; no efficiency/performance/scalability guide discussing when certain language features should be favored over others given the current way they're executed (e.g. is IF-ELSE, CASE-WHEN, or a series of IFs with early returns cheaper? when should I favor using FOR with a query, vs. when should I query data into an in-memory array variable and then use FOREACH, vs. when should I query data into a TEMPORARY table and then query that?); no place where you can get a sense for how procedure CALLs interact with MVCC (e.g. when they acquire + release locks, and therefore how and when they cause blocking on contended tables vs. how and when a SELECTed function that uses dblink/fdw to run independent txs would do so); etc. There isn't even a single mention of which PL/pgSQL exceptions are potentially raised by what PG builtin functions when called in a PL/pgSQL context; how to name those exceptions to match on them to catch them; or how to raise them yourself. I often need to dig into the PG source code to figure that out! (PL/pgSQL honestly feels, in docs terms, like a proprietary third-party language-engine "plugin" that someone bolted on, where the docs were expected to be provided by the third party, but never were. But it's not! It's a first-party language, and the reference implementation of how to create a language extension!)