Composition shouldn't be this hard

Your timing for this effort is good because there is going to be a lot of thought leaders who are going to stop putting in the effort given the deafening avalanche of slop and devaluing of reusable code units in the wake of AI. So, less competition to novel/original efforts in the "operating environment" space like yours.

Your timing is also good because preeminent OS designs are becoming less relevant to the hardware of today and so a new operating environment could include holistic designs with abstractions that could reach all the way back to the system layer. Perhaps a good approach could be to bootstrap off of existing operating systems and containers and work your way back to the metal. We really need a modern Java VM-like effort that includes a memory model with the ambition to include I/O buses and sidecar computing devices like GPU/TPU/NPU.

Your message is a noble "boil the ocean" one, and looking at past efforts like BeOS/BeBox or modern things like Oxide Computer Company, I feel like shipping a boil the ocean project is possible for the pros, especially with enough runway. Perhaps boiling one sea at a time is possible? ;) Just don't paint yourselves into any corners.

In my personal experience, microservices development has been a successful way to develop software while addressing some of the versioning challenges you mentioned. I managed the impedance mismatch between API versions through JSON schema validations and allowing deprecation periods for old versions by hosting multiple version endpoints to the same backend to allow callers grace time to upgrade. Requests payloads are validated against their JSON schema at runtime, and response JSON payloads are tested against their JSON schema during integration testing. Runtime is "too late" to know when things are wrongly connected as you said, but at least I know loudly and don't accept crummy data.

Your discussion made me think of Alan Kay's discussion of how objects in a perfect world should be able to pass messages between one another: as a negotiation to come to an understanding rather than as rigidly defined payloads. I think it was a fanciful idea, but in the age of AI, such a loosey goose negotiation process seems attainable if it weren't for hallucinations. Active interrogation and negotiation is the key to combating impedance mismatches prior to runtime.

Finally, consider software supply chain trust issues. If you're going to boil the ocean, you gotta do much better than we've done. The internet was designed in the age of peace and love. We've reached the "we're cooked" age, please send help!

In any case, I hope you take my message here as encouragement, best wishes and I hope you succeed beyond measure.

You can also run a successful manufacturing company without considering environmental impact. A successful tobacco company without considering public health. A successful social media persona without considering cultural impact. A mobile app slop farm that floods the app store. If economics is your priority, then everything comes down to that.

I think this vision is also shared by Unison: https://unison-lang.org/

The way I see it, Clojure was created to address most of the philosophical problems highlighted in the article, and its ecosystem has developed under the influence of such ideas.

It embraces the dynamic nature of things. Every program that must do I/O is dynamically typed at the edges - so statically-typed languages must have both the facilities for dealing with dynamic types and static types, and use those differently for "inner" and "outer" parts of the program. The Clojure ecosystem makes it more approachable to work with dynamic data, and as a result, program's parts (modules, even functions) use the same machinery as the program as a whole. What helps, off the top of my head:

1. Immutability of values, persistent data structures, explicit state management - it turns out most of the "problems" with dynamic types are not due to the fact that types are unknown, but the fact that they may change over time. Most of concurrency problems also become irrelevant with immutable values.

2. Powerful data specs/schemas allowing to express full business requirements and constraints on expressible values (not possible with static typing anyway) — making invalid states unrepresentable and actually meaning it.

3. Concise unified model for working with different logical data structures, also nested data structures.

4. Simple asynchronous communication mechanisms and patterns between independent modules of a programs.

5. Rich extensible aggregates of data with namespaced keys as the object model.

6. An ethos of extreme backward compatibility (and tools for achieving and maintaining it) through accretion of novel values/parameters/features, enabling reloadable workflows, which makes it easy not only to redefine functions in code while program is running, but also updating individual programs while interconnected system is running, without breaking things.

The downside is that tools that reduce complexity are not very easy to use for people with less experience, so they cannot be too popular. People tend to overcomplicate, and to be quick to dismiss well-thought-out solutions.