Sorry for a bit of a blunt comment following -- but your rose-tinted view of even the original Agile / Scrum ticked me off. I have to push back here, severely so.
> The Daily was fantastic because it was lightweight and short, but very frequent, so that communication flowed freely.
You should qualify your statements with "for me" and "for our project" because dailies have not been a net positive over my 23 years long career. Yes, not even once. I can't remember a single time I enjoyed them, nor a single time they ever helped me with anything at all related to work.
I am not introverted, nor autistic (though I very likely have ADHD). I am quite outgoing in fact, yet I will hate the dailies until my grave.
The only thing they achieved was put some juniors back on track because when they get blocked they also close up and don't talk to anyone (for whatever reasons that I'll never understand apparently), and give excuse to introverted people to open up a little and have some casual chat. I am not against the latter but I dislike work meetings being hijacked and turned into half therapy sessions.
I've suggested to them to do periodic screen-share pair-developing sessions, they did it, they absolutely loved it and kept doing it even after I left, and us who didn't want to do casual chats in supposed work meetings enjoyed the work meetings slightly more. Everybody won.
> The Retro was the catch-all meeting to bring up stuff (People forgot just how big the obstacles were back then - I remember having to wait 12 weeks for a config change that allowed my app to connect to the database).
And again, please add "for me" and "for our project". Retrospectives have been used in my career, without failure, without a single exception, to slap developers into rushing even more. That's what all managers I was ever under viewed them as: an opportunity to "correct velocity".
Masters whipping up the slaves because they don't pick cotton quick enough. Try and sugarcoat it as much as you like -- it's that and it was always that, and the people in power will always try to swing everything in that direction. It's who they are. It's what they are.
> Recognising that one person who always tried to bring everyone together, calling them a Scrum Master and making room for them to do these tasks was a no-brainer.
Really cute, until you had my life and career and saw the "scrum masters" being one of the managers cousins who saw an opportunity to give them income while pretending they are useful.
In my defense, I never witnessed a managerial system that helped me, so bear with me here.
> "We're all adults here, we don't need these rules" was a wide spread sentiment, yet work was horribly inefficient.
And for the third time: maybe in your teams. I worked in no less than 6 teams that did this very, very well. To the point of us not needing a manager because I and one other guy (we were 7 in total) basically said "OK, things A / B / C are more or less done but we still need X and Y; me and Joel are busy with infra stuff, anybody wants to pick those up?" and somebody always stepped up.
Is that what a scrum master is supposed to be doing? I've never seen it though. But we managed to distribute load and responsibility between ourselves pretty well.
Predictably, that team was eventually disbanded because we had actual power during the executive meetings (me and the other guy attended them). Nobody likes programmers who can push back in an informed manner that ruins the CEO's prepared graphs and slides. Who wants their beautiful illusions shattered by facts? Not these "adults" for sure.
And yes all of us left shortly after. And yes 3 out of the 7 of us were called back some months later to fix the messes of the others. We beat the code back into shape in less than a month, charged them triple and laughed our way to the bank.
---
Bigger point is: we all know the beautiful theory. But there are people out there who don't like it and want to skew the practices to what serves their interests, not yours and not mine.
Glad that you had such a positive experience. Really. But you should be more objective and remind yourself that you are very likely privileged. Most of us are not.