So totally agreed that rewrites are almost always not a good idea, as per the famous "Things you should never do!" Joel Spolsky post (
https://www.joelonsoftware.com/2000/04/06/things-you-should-...).
The difference for a project like Matrix however is that our product is not really any single software codebase but the protocol spec itself. In fact, it's a bad smell about the spec if it's too hard to implement it - whether that's clients or servers. So part of the point of the Dendrite (golang server) project is to dogfood the spec and fix its various shortcomings (as per the OP's complaints) and make sure it's fit for purpose. For context, the core Matrix team has already written 4 entirely disjoint Matrix client codebases (Matrix Console on Web, Riot/Web, Riot/iOS and Riot/Android) as part of developing the spec and ensuring it's fit for purpose, so it's not that unreasonable for us to also write 2 server implementations (3, if you include the failed Dendron project).
In terms of blog posts: https://matrix.org/blog/2017/03/15/dendrite-receives-its-fir... is probably the most revealing one.
It's worth noting however that Dendrite progress has not been entirely smooth, though. Particular problems have been:
* When we lost funding last year (https://matrix.org/blog/2017/07/07/a-call-to-arms-supporting...) it inevitably had an impact on the team - and ironically the only two people who left the project ended up being the two folks who were working on (and created) Dendrite. This genuinely wasn't a reflection on Dendrite itself (which is ironically one of the most fun projects to work on in Matrix, given it's all blue skies and green fields in terms of implementing a server that learns from Synapse's many mistakes), but more a sad coincidence. Either way, it ended up with the project being under-resourced (although we're finally fixing that during May having re-hired :)
* Matrix traffic accelerated massively during 2017, and Dendrite wasn't yet ready to use in production - but meanwhile Synapse wasn't performant enough to keep the matrix.org server running smoothly, so we had to pull folks off Dendrite in order to keep Synapse scaling (ironically, ending up taking inspiration to some extent from Dendrite in improving the Synapse codebase)
* In general, it's hard to make progress on an R&D project which isn't yet in production when there's an existing project in production which is on fire.
* We deliberately scoped Dendrite to implement a server against the current Matrix spec that Synapse implements - rather than trying to improve the Matrix spec at the same time. This is good in terms of feature creep, but possibly a bit demoralising as you find yourself failing to innovate and improve the spec and instead re-implementing it, warts and all. I'm not sure what the right balance is here, and in practice we've ended up with the Matrix spec evolving significantly under Synapse, and meanwhile Dendrite then becomes a moving target as a result.
* We've almost certainly fallen foul of the Osborne Effect (https://en.wikipedia.org/wiki/Osborne_effect) too, where folks have held off on using Matrix or running Synapse because they're waiting for Dendrite to land, RSN.
But on the plus side, there's a small but active FOSS community contributing to Dendrite - much more than we've ever had on the relatively impenetrable Synapse codebase. And Dendrite itself works (about 80% of an MVP complete) and is demonstrably ~2 orders of magnitude faster than Synapse (even whilst Synapse's performance is also speeding up). And as we find algorithmic breakthroughs in performance as part of improving Synapse, these breakthroughs equally apply to Dendrite and will make it even crazier fast.
So, I think it's too early to say whether this is a textbook example of "second system syndrome", or whether instead it's a mature and sensible approach to ensuring a heterogenous ecosystem of implementations for an open standard and dogfooding one's own protocol. My hunch is that if we hadn't hit funding nightmares last year it would be a no-brainer win; but even now, it's likely to eventually work out for the best in the end. The best analogy is not the excruciating migration from Netscape to Mozilla that Joel wrote about in 2000, but the more spectacular and successful migration from Gecko to Servo happening in Mozilla right now. It's a massive massive architectural improvement and rewrite in a new language which requires a lot of work, but pays off in the end.
(This is my personal take on it, at least, as project lead for Matrix; those actually working on Synapse & Dendrite probably have different viewpoints! :)