Yes I did feel the same, and it boiled down to accepting I had to try something new that I thought I didn’t want or be happy with what I already had . You have several options I guess.
- it’s very likely you haven’t mastered all layers of the stack - you maybe have mastered the parts you’ve come across though. Would you include Linux kernel development, program proofs, HPC , math-oriented software, compilers, firmwares, fpgas, database internals ( I mean, writing a full database engine ), big iron infrastructure , actual research etc in the list of things you’ve mastered? That’s what I did for many years ( not necessarily these topics ) - keep exploring , the field is massive. In my experience, the more you look around the more you realize that you haven’t seen it all, and even less mastered it.
- now, in my case, after doing just that ( trying new technical things) I realized I was getting increasingly interested in the overall technical strategy and interplay with people more so than the purely technical side of things. I had vowed I would never get into the higher spheres … but I am now essentially a principal engineer ( or whatever you call that but basically I lead some technical stuff ) for a very large org. This is not management , and I really like it. I keep learning about people and business, about how changing something big is hard and human , and balancing technical problems with human ones . It’s fascinating, and above all grounded in technical knowledge so I don’t feel like I’ve thrown away a significant chunk of my career.
In particular, if you feel that you’re not being listened to but would like to, maybe you’d be interested in that kind of job. If you’re saying you’re not always succeeding at persuading people , it probably also means there’s growth for you here, and satisfaction down the line.
- actually get into management , which like most topics is actually interesting once you get into it. It’s just different. I thought I didn’t like it - I actually do and that was a colossal surprise to me.
Whether it’s technical leadership or management, these skills can be learned, and they make the bits that you don’t like much more palatable, and the bits that are ok very enjoyable. I was lucky enough to meet very good people who taught me properly. I think this is very important. I can cook at home, but if I get proper training my food will be better and I will enjoy the process a lot more ( try chopping vegetables with no training - you’re slow as hell and it ruins everything)
- do something unrelated ( cooking etc ). To be honest, at some point I did my job but the more fulfilling exploration part of my life was at home and revolved around food. I did consider changing career completely but decided it was too risky.
To a large extent, there is limited value to your extensive knowledge if you remain at the bottom because it will always have limited impact. You seem to want the power and influence ( for good reasons !), but don’t seem to want to learn the skills that are required . ( at least management, you seem interested in technical leadership). I guess you have to choose. It’s very hard, but liberating.
You mentioned family - I also made sure I was in a good place at home when I switched. I found it hard but support from my wife helped tremendously.
Best of luck !