I can appreciate that you guys weighed the tradeoffs and decided that giving users freedom to override arbitrary styles is worth the cost of whatever breakage might result from that decision as you upgrade the library. We're all engineers who build things to solve real world problems at the end of the day, and I don't have any of the context that led up to your decision, so I can't say for certain that I'd have weighed those tradeoffs any differently if I were in your shoes and have to cater to the whims of thousands of engineers with differing opinions on how things should be done.
However, I have to disagree with your characterization of render props as "the freedom to completely replace the guts of your component". Offering a render prop API should be an explicit decision to fundamentally stop treating that branch of the render tree as part of the "guts of your component". It's a decision to delegate to users on how to best render that part of the component.
Of course, you're right that in a vacuum, that would equate to throwing up our arms and asking users to figure it all out on their own as to how to implement the styling and functionality of that part of the tree.
However, when we're the maintainer of a component _library_, we're in the unique position where we can provide additional, complementary components that provide styling and functionality for users to use to implement that part of the tree, composed with their own custom components when appropriate, without having to build everything from scratch.
These components usually start out as the same components that used to reside in that part of the tree in the original parent component. By decoupling them from the parent, they're then immediately able to start providing their own explicit interfaces that can be evolved independently from the parent without risk of breaking usages on implementation detail changes.
Of course, users are free to simply not use those components because they might not address whatever problem they need to solve. Rather than taking that as a failure of the approach, I'd take that as a triumph because it demonstrates that this approach gives users the flexibility to experiment with how best to solve their particular problems within the confines of the isolated subsection of the original component without affecting the maintainability of the component itself. And we as library maintainers are then able to examine the various custom components created for those use cases to see if any particular implementation is suitable for extracting directly into the library, or at least learn from them when building new reusable components to support those use cases officially (and users are free to choose to adopt those new components at their own pace, without any fear of things breaking under their feet).
This is why I point to this approach as the middle ground. Component composition is a much more sustainable mechanism to provide to users for customization, in my opinion, compared to arbitrary overrides. In fact, if anything, I'd consider arbitrary overrides to be the nuclear option here, because once we start offering that option, people are going to start using our components ways that we can't possibly ever fully anticipate, so we end up having to _really_ throw up our arms as maintainers and start saying things like "We don't consider changed styles as a breaking change".