Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Opposite of Documentation is Superstition (2020)

buttondown.com

Post reply on HN
Print view View original

41–50 of 87 posts

Re: The Opposite of Documentation is Superstition (2020)

#41

Post by floating-io »

vincent-manis wrote: post #13

I can't remember the number of times I have given up on software because there is no “Concepts and Facilities” document that gives me a mental model of what the program will do and how I operate it. Instead, one sees a website with a mass of unrelated documents that I'm supposed to read in some order, and divine how everything works. Maybe because I'm a child of the 20th century that I value documentation that gives…

Not just an issue for documentation, but also for marketing materials. I can't count the number of times I've looked into some app that was referenced vaguely in a comment somewhere, only to have to dig for half an hour on that thing's web site just to figure out what the thing is supposed to even do in the first place...

Re: The Opposite of Documentation is Superstition (2020)

#42

Post by lelandbatey »

IgorPartola wrote: post #20
Animats wrote: post #15

I've called this ritual-taboo programming for decades. It happens for user interfaces and APIs when the documentation is absent, or only consists of examples. If there's no reference documentation, everything is a copy of something someone else did. Nobody understands how it really works. Now, for some interfaces, this isn't too bad. Most people don't know why US AC power plugs are polarized, or what the ground prong…

How do you view self-documenting interfaces? Have you ever read the manual for your browser which you are using to make this comment? The OS on which the browser is running? The device on which they are running? I think there are interfaces that need to be usable assuming documentation will not be read. These are of course user interfaces, not necessarily APIs. But if you are creating something like a library of basi…

> How do you view self-documenting interfaces?

> Have you ever read the manual for your browser which you are using to make this comment? The OS on which the browser is running? The device on which they are running?

Have you ever read the manual for the tools used to build the building that you live in? A manual for the trowels, tape, mud, and drywall that built and makes up your walls? Or a manual for the Romex in those walls? A manual for the concrete foundation of your building?

No? That's because using/depending on those things doesn't require nearly anything of you; you stand on the floors and lean on the walls without thought.

You browse the web constantly, so you do it without thought. Not so with your taxes, despite their similar constance and ubiquity, because they are not constant and thus haven't had the same abstractions engineered into their very DNA.

But even things like buildings/floors/doors with that level of "no need to think about them" only afford that to you based onto their frequent use. Few other pieces of software afford that same consistency. With time though, they will. I think we'll see a lot of software grow to be everore of a "skin round a spreadsheet", especially as we comoditize the few remaining frontiers of UX design (e.g. mobile devices) in the same way as desktop design has converged on common patterns like the chat-log-and-history, the search bar, the photo gallery, the text editor, and the spreadsheet.

Re: The Opposite of Documentation is Superstition (2020)

#43

Post by sahmeepee »

IgorPartola wrote: post #20
Animats wrote: post #15

I've called this ritual-taboo programming for decades. It happens for user interfaces and APIs when the documentation is absent, or only consists of examples. If there's no reference documentation, everything is a copy of something someone else did. Nobody understands how it really works. Now, for some interfaces, this isn't too bad. Most people don't know why US AC power plugs are polarized, or what the ground prong…

How do you view self-documenting interfaces? Have you ever read the manual for your browser which you are using to make this comment? The OS on which the browser is running? The device on which they are running? I think there are interfaces that need to be usable assuming documentation will not be read. These are of course user interfaces, not necessarily APIs. But if you are creating something like a library of basi…

> Have you ever read the manual for your browser which you are using to make this comment? The OS on which the browser is running?

For niche situations, yes, e.g. different browsers having different keyboard shortcuts for hard refresh. It's quite common for people to assume they know how to do it in one browser because they knew the shortcut in another, but when they just get a normal refresh they don't realise their error.

Software written to be used by billions of people per day must be written to have a very low barrier to entry, and even if it contains sophisticated and complex features they must not be essential for >99.99% of the software's overall usage.

The downside of this approach is that many people never learn about features in everyday software that would make them much more productive. Keyboard shortcuts as above are the classic example and are almost always documented, but there are many more unexplored features and not all requiring significant memory or technical skill to make use of. I've not heard of a regular office job in the last 15 years where there would be induction training on the core tools (OS, browser, office suite). It's assumed that you're not employable if you don't know how to use them, and by implication that knowing how to use them is a binary thing rather than a spectrum.

Re: The Opposite of Documentation is Superstition (2020)

#44

Post by brabel »

IgorPartola wrote: post #20
Animats wrote: post #15

I've called this ritual-taboo programming for decades. It happens for user interfaces and APIs when the documentation is absent, or only consists of examples. If there's no reference documentation, everything is a copy of something someone else did. Nobody understands how it really works. Now, for some interfaces, this isn't too bad. Most people don't know why US AC power plugs are polarized, or what the ground prong…

How do you view self-documenting interfaces? Have you ever read the manual for your browser which you are using to make this comment? The OS on which the browser is running? The device on which they are running? I think there are interfaces that need to be usable assuming documentation will not be read. These are of course user interfaces, not necessarily APIs. But if you are creating something like a library of basi…

> But if you are creating something like a library of basic math functions I would rather read your code than your docs.

Talking about software specifically: this is the exact wrong way to do it. A programming interface should be easy to use and "self-documented" to some extent (by, e.g. using the type system as much as possible to make bad states unrepresentable), but only very simple interfaces are usable that way. For most real-world interfaces, there's just no way you can encode all the "rules" in the type system and the names. You absolutely need to document how your interface works in general, what each part is used for and expected to do, special cases and so on that are just impossible to know otherwise. Now, if you just read the implementation instead (and notice that you would need to read all implementations, most interfaces are likely implemented in many different ways) you're not coding against an interface at all. You're assuming the implementation is the interface. You're ignoring what the implementer intended to be private and subject to change. Don't do that.

Re: The Opposite of Documentation is Superstition (2020)

#45

Post by brabel »

codr7 wrote: post #26

Isn't this more likely an example of LLM-crappification? As in, the reason it isn't documented is that no one has a clue...

In that case, it should be documented at least that the process is driven by LLM and hence, it's not known exactly how it makes a decision...but I guess that would not look good to most users so they just leave it out.

Re: The Opposite of Documentation is Superstition (2020)

#47

Post by pona-a »

iterateoften wrote: post #5
nox101 wrote: post #4

Seems to me documentation can also be superstition if it's out of date. I wonder if LLMs can generate docs at any reasonable level of usefulness and accuracy Speaking of which, I just tried using ChatGPT to create a React app with typescript and rollup. What it generated didn't work (got errors). Of course I could tell it the errors and it fixed them but it suggests they aren't yet ready to write accurate docs

Maybe LLMs make docs irrelevant if they are just generating static pages or doc strings. Why spend the time generating something static when everyone has their own LLMs they can just ask directly? Anything you can generate as docs someone can generate themselves anyways.

The thing is, good documentation captures the difference between what you expect and what something is. LLMs, as they exist now, are mass-producers of cliché. You can sense it even in their prose or those inverted puzzle evaluations [0] , they are simply statistical models picking the most likely options without enough depth to subvert the average. When you ask them to describe a piece of code, you'll only get the rephrasing of what the code already says.

- [0] https://github.com/cpldcpu/MisguidedAttention

Re: The Opposite of Documentation is Superstition (2020)

#48

Post by tempodox »

Animats wrote: post #15

I've called this ritual-taboo programming for decades. It happens for user interfaces and APIs when the documentation is absent, or only consists of examples. If there's no reference documentation, everything is a copy of something someone else did. Nobody understands how it really works. Now, for some interfaces, this isn't too bad. Most people don't know why US AC power plugs are polarized, or what the ground prong…

It's always the same problem: The work generated by doing documentation, including the possible need for specialized talent, is quite directly perceivable. The work generated by the absence of documentation is more under the radar and disguised as “cost of doing business”, which makes it harder to convince the anti-documentation crowd. Not to mention the “job security” aspect of everything being in the heads of the ICs.

Re: The Opposite of Documentation is Superstition (2020)

#49

Post by martin-t »

nox101 wrote: post #4

Seems to me documentation can also be superstition if it's out of date. I wonder if LLMs can generate docs at any reasonable level of usefulness and accuracy Speaking of which, I just tried using ChatGPT to create a React app with typescript and rollup. What it generated didn't work (got errors). Of course I could tell it the errors and it fixed them but it suggests they aren't yet ready to write accurate docs

I inherited 100k LoC "documented" by an LLM.

LLMs can put into words what a programmer can infer from struct definitions, function signatures, names and even taking a cursory look at a function body.

LLMs cannot retroactively find out the _why_ of design choices. If you don't document the reasons behind decisions while making them, they'll be lost, which is exactly what leads to superstition. As a result, changing code is error prone because you never know when code is written in a particular way for a good (and still valid) reason or just because.

Re: The Opposite of Documentation is Superstition (2020)

#50

Post by smitty1e »

worksonmymach wrote: post #35
smitty1e wrote: post #2

I have also used the phrase "tribal knowledge" in the documentation-vacuum context.

Hard to avoid some tribal knowledge. To be free of it you need full reference documentation and tutorials for everything you do. And to update it as you go. And to realize what might be missing from a paedagogical perspective. Not easy and probably not possible for a commercial operation. And not necessary. 80-20 rule applies.

And applying that 80/20 rule implies a culture of writing things down and communicating them, which doesn't scale and is highly dependent upon the general chattiness of the team in question.
Post reply on HN