The problem with having standards for this kind of thing is that different units have different needs for communication and different levels of being smart. For example, some units want 2 temperature sensors and some want 3. The method used to control the system can be relatively complex - some systems are using physical models of the characteristics and positioning of sensors to do fancy control, and there are probably at least 5-15 data points involved in a typical system.
While it would be nice for the protocol to be documented (would realistically only be used by a very small number of users), the only real way you would be able to get a standard for something like this to work is if you went the Bluetooth route and did generic scenario-based profiles (e.g. HFP, A2DP, SPP), and optionally some "GATT" or "generic attribute" parameters. However, as we see with Bluetooth LE, everyone just uses GATT and implements their own little proprietary thing over it and you're back to the same problem.
Some of these systems attempt to be "smart" and just use the 24V C/W/Y1/Y2 etc protocol as a "standards compliant fallback". You don't necessarily lose ALL of the smarts, but the unit has to essentially use physics magic to make an educated guess about the information (for example, if you use a on-off thermostat, you can't really measure the temperature of the setpoint, so you don't know how close you are unless you somehow make an observation over many cycles.
I think that reasonable attempts to address this problem could involve some kind of extension to the old 24V interface - say, by offloading the actual "policy" part of system control to the "thermostat" i.e. have something that goes from 0-10V where 5V is off, 0V is full cooling and 10V is full heating. This allows you to choose your own temperature sensor situation, but complicates setups where more than one zone or thermostat is required. Of course, it will be very difficult for the industry to settle on a solution to this. Qualcomm's Quick Charge 2.0 was a very simple protocol similar to this, which was essentially self-documenting and not something that needed versioning, but of course, needs changed, 3.0 came and went, 4.0 came and went, and by the time USB C and USB PD came around you ended up with a full on data protocol API with all the OSI layers and of course, vendor specific extensions.
You could define some complicated protocol where you don't conform to a standard but you publish an API for your system (of course, there is no incentive to do this), and larger vendors like Control4 or Lutron, Crestron can program their products to interface with it. Unfortunately this doesn't allow the customer full choice over thermostats, because now you have to deal with N vendors x N thermostat vendors, which isn't scalable and you'll end up in dependency hell.
The closest thing I can think of to a standard, and the way it is solved in larger buildings, is through something called BACnet. It appears to use the Bluetooth model of "scenario based profiles", with all of the disadvantages that come with that, but the primary disadvantage is that it has to be to some degree manually configured to route data where it needs to go - and I don't think this is something installers are currently equipped to do at home scale.
Realistically, the "thermostat" is just a vestigial component in modern terms and really, it's just a user interface and thermometer now. Without getting into the wish to have open sourced app control or whatever, it's hard to define what the "thermostat" does and what the "system" is doing, and whether the device that sits on the wall is really a "thermostat" deserving of being interchangeable anyway. I have heard from a friend that does home automation integration that many clients don't like the default thermostat because it doesn't look very aesthetically pleasing. In this case, I'm definitely sympathetic to the need for customizability but it seems difficult to achieve in practice.