An Experiment on Code Structure

I wish for a future where we can have more than one concurrent view of the same code. Structure need not be derived from from mere files and newlines and a handful of semantic organizational elements (function, class, module).

The current way of doing things forces us to make a compromise between prioritizing the forest over the trees, or vice versa. Programming languages are largely concerned with the trees' bark. But to make good software, you need to see and understand both, so the compromise is always a problem.

The solution probably needs large-scale re-imagining of how compilers, languages, version control, and editors/ides work (which also requires one to accept that working with a simple flat-file text editor won't work -- a bitter pill to swallow for someone like me who likes the simplicity of simple text editors).

I have some (very vague) ideas, but gosh, how do I find the time to experiment and refine or reject them...

Future programming languages will be graphical in one way or the other, I think. As you said, the programmer needs to have a clear way of visualizing the big picture. I think this can be achieved without forcing people to go visual. You could have the code on the one hand, and the metadata for the presentation of the code on the other, in a separate file. You could also just hide the graphical metadata for the code view.

Separate metadata/markup for presentation + code sounds sounds like a straightforward choice, but I'm concerned that it'll incur a lot of maintenance overhead, and the programmer working with the code still needs to keep it up to date and relevant somehow. Dunno, I feel like it'd feel like code + doxygen boilerplate comments (a pain in the butt if you ask me) but worse.

I'm thinking that we need language level support for higher level semantic constructs and relations. Right now code is somewhat analogous to raster graphics or very simple vector graphics. You can construct anything with it, but it is very rigid and there's only so much high level structure that tools can try to infer and dump out of it. (Think call graphs, dependency graphs, flow charts, index of class hierarchies.. all of them somewhat useful for certain purposes, but none of them really good for high level design work or reasoning about systems at a level above the plain code).

We could slap some metadata on vectors or raster images but I think that's a far cry from ideal. I think that, with sufficient support from the language, we can provide most of the visual structure for alternate views by simply graphing with help of the semantics that are laid bare in the code. I wouldn't mind some additional hints for presentation, but if we're adding lots of markup and metadata, I think we're going in the wrong direction.

I love this. Currently working on a file storage system that gets away from folders, and that's hard because everyone has folders hard-wired into their brains because history.

Functions shouldn't live in files, for a start. Files are an artefact of storing code in a file-based storage system, and have nothing to do with code architecture. Creating a code editor that stopped working with files and only worked with functions would be interesting as a start on this, I think...

I've become a big fan of not worrying about architecture until the rewrite. The first version is always an exploration of the problem domain, and treating it as that has always made my projects go quicker.

This is going to trigger some people, so here's some caveats:

- there's always a rewrite. Even with perfect architecture. Usually because nobody understands the problem domain until there's been an exploration of it with a first attempt (occasionally for other reasons). A few have two rewrites. And that's not a bad thing. Starting again with better knowledge can make the whole project go quicker, because there's less chance of ending up in the situation TFA talks about ("we have to refactor because tech debt").

- architecture needs to be shaped by the problem domain. There isn't a "best" architecture, so picking one requires knowledge of what the code needs to do. And that needs an understanding of the problem. No-one understands the problem from a technical point of view until/unless they've tried writing a program to solve it.

- a lot of features of architecture (like choosing to DI the database engine, instead of picking an engine because it's clearly the right choice) are made because the devs don't have enough knowledge to make an architectural decision when they write the code. It's interesting to see how many of these disappear on the rewrite. It's always more efficient (both performance and development time) to make these decisions, but making them is difficult without enough problem information.

- never underestimate the power of a monolith with good file structure.