>> There must be a middle ground where developers can focus on the core business logic that yields the most value without incurring technical debt and making the development process a nightmare. I don’t have an answer for that, nor have I worked at a company that found the perfect balance. Plus, I’m not a technical lead, manager, or business owner. So if you are one of them, I’d love to hear how you or your organization plans to tackle this
I'm a technical lead, business owner, and ex-manager. So I guess I qualify to answer :)
The short answer is that we have only ever had a very (very) small development team. Typically around 3 or 4 people.
Secondly we're using "old" tech, nut with up-to-date tooling. Our product contains code written in 1996 and iterated on since.
I'm lazy, and I'm not interested in writing it all again in some new language. So we haven't done that. I'm lazy, so I'm not interested in re-architecting it every 5 minutes. I'm lazy so i prefer simple, maintainable, easy to read, code over cleverness.
Most of all, as a startup (boot strapped) I didn't get paid in we didn't make sales. (That happened a lot in the early years.) So shipping is a priority. Business is a priority. But since I'm lazy I avoid solutions that'll break things, that'll cause unnecessary work in the future.
I have enough autonomy to dictate pace of delivery. I have enough incentive to move the business forward (in the short and long term.) I don't need to resume pad (this is the first ,and last, job I'll ever have.)
Alas none of this advice is transferable. What works for me likely won't work for you. Our context is likely too different.
So yeah, there are businesses in between the extremes. Ones with competent developers. Ones either competant managers. Ones where all the business interests are kept in balance. They are not easy to find. Good luck.