> software developers always want to have the freedom to “improve” their own software
Developer can still have that freedom in most[1] situations. When you have a new idea for a better/faster implementation that is, unfortunately, incompatible with the existing interface, you can usually provide compatibility with a shim that implements the old interface using the new interface.
Sometimes this is little more than some #define renames and a handful of very-thin wrapper functions. Occasionally it will be a bit heavier, which might make the legacy interface slower, but this should be balanced (approximately) the faster underlying implementation. If the compatibility shim is significantly slower (or has other unavoidable runtime issues), an upgrade that is slower for legacy uses that still works is always better than breaking existing code.
For an example and opinionated discussion, see JWZ's post[2] about his OpenGL 1.3 -> OpenGL ES 1.1 header.
[1] Exceptions are cases where there is a *fundamentally unavoidable cost to providing the interface (e.g. where N is the total number of interface entry points, something in the critical path unavoidably MUST take >O(N) time or space, or N is limited externally to a small value ("we only have silicon for at most N instructions"))
edit: [1b] Another exception would be when the legacy interface itself is a serious security problem. If your legacy interface unfortunately forgot to include a way to supply credentials and simply accepts writes from anyone, a prompt fix that breaks that API is necessary and appropriate. Next time you're designing an interface, be sure to remember that security has to be baked in from the very beginning!
[2] https://www.jwz.org/blog/2012/06/i-have-ported-xscreensaver-... (you might want to copy/paste that URL if your browser betrays where you came from with a Referer header)