A year of Rust in ClickHouse

One thing I often see pop up in larger projects, which in the article is likely the fault of way to large symbols, is overuse of generics/type state/etc.

Or you could formulate this as needless obsession with not using `dyn`.

And sure generics are more powerful, dyn has limitations, etc. etc.

It's one of this "Misconceptions Programmers believe about Monomorphisation vs. Virtual Calls" things as in:

TL;DR: dyn isn't as bad as some people make it out to be; Weather perf. or convenience it can be the better choice. Any absolute recommendation of always use this or that is wrong.

- wrong: monomorphisation is always faster; reason: monomorphisation pollutes the instruction cache way worse, as such in some situations switching some parts (not all parts) to virtual calls and similar approaches can lead to major performance improvements. Good example here are various experiments about how to implement something like serde but faster and with less binary size.

- wrong: monomorphisation was picked in rust because it's better for rust; right: it was picked because it is reasonable good and was viable to implement with available resources. (for low level languages it's still better then only using vtables, but technically transparent hybrid solutions are even more desirable)

- wrong: virtual calls are always slow in microbenchmarks; right: while they are more work to do modern cpus have gotten very very good at optimizing them, under the right conditions the might be literally as fast as normal function calls (but most times they are slightly slower until mono. trashes icache too much)

- wrong: monomorphisation is always better for the optimizer; right: monomorphisation gives the optimizer more choices, but always relevant or useful choices but they always add more work it has to do, so slower compiler times and if you are unlucky it will miss more useful optimizations due to noise

- wrong: in rust generics are always more convenient to use; right: Adding a generic (e.g. to accomodate a return position impl trait) in the wrong place can lead you to having to write generic parameters all through the code base. But `dyn` has much more limitations/constraints, so for both convenience and performance it's a trade of which more often favors monomorphisation, but not as much as many seem to believe.

- wrong: always using dyn works; right: dyn doesn't work for all code and even if it would using it everywhere can put too much burden on the branch predictor and co. making vcalls potentially as slow as some people thing they are (it's kinda similar to how to much monomorphisation is bad for the icache and it's predictors, if we gloss over a ton of technical details)

So all in all understand what your tools entail, instead of just blindly using them.

And yes that's not easy.

It's on of the main differences between a junior and a senior skill level.

As a junior you follow rules, guidelines (or imitate other) when to use which tool. As a senior you deeply understand why the rules, guidelines, actions of other people are the way they are and in turn know when to diverge from it.

You'll get loud, obnoxious idiots in any big enough crowd. Also [source needed].

Do you have a specific snippet in mind which demonstrates the issue? It is likely more of a team or a community issue when it comes to writing unreadable code than a language one since it tends to happen in every sufficiently powerful language.

C# did not “have to absorb every idea from F#”. This is not how programming language development works. You can read LDM notes at https://github.com/dotnet/csharplang/discussions?discussions... and specs are documented in the repo.

> rather than making F# a viable programming language

F# is a viable language aside from using specific few libraries that don’t play with it nicely or around writing ref struct heavy code. I’m not sure what makes you think it is not. In comparison, it is probably more viable for shipping products than Scala, Clojure, OCaml and Haskell.

> So you’re using formal methods, sanitizers, the whole gamut of verification and yet you can “quickly” find new issues just by fuzzing. Sounds like there’s some significant problem there that you’re not mentioning.

Or perhaps you're missing the super-text, that all those things were insufficient to make C safe.

> The problem with this is if you have a team working on a C++ product you will need some people who can catch memory bugs to review every code before merging. Even with this approach it still possible to missed some memory bugs since the reviewer need to fully understand each object lifetime, which is time consuming during code review.

Nah, if you're trying to match every "new" with a "delete" during the code review, you've already lost the battle. You can probably succeed when the code is added, but then the edits start to flow and sooner or later it's gone. Reviews are mostly good to catch design problems, not bugs.

The only reliable approach I know is to have a strict rule of never mixing memory management with business logic. Nothing else works well enough but this one however works remarkably well.

Business logic should rely on containers, starting with simple unique_ptrs and vectors and going deeper and deeper into the custom land when appropriate. If you can't find a suitable standard container, you build a custom one. The principal difference of "writing a custom container when you need it" compared to "integrate custom memory-management into the business logic when you need it" is that containers are:

* well understood

* well tested

* relatively small code-wise

* almost never change once implemented

None of the above applies to the business logic, it's the complete opposite.

Think of it kind of like programming in Java: someone has to write the memory management and it's a hell of job. However once this is done, programming the ever-changing business logic is easy and safe.

You can live the same life in C++ AND also have the ability to put on the "Doomguy of the memory management" shoes whenever you feel like it. Just don't forget to take of the shoes of the "business logic guy" when you do it, you can't wear both at the time.