One thing I'd like to see is native support for some notion of pitch other than 12-tone equal temperament. Looking at their example, I see this: ...which looks like it's basically just a representation of the underlying midi. One way they could represent other pitches is for "key" to allow a floating point value rather than an integer. So, for instance, 65.5 would be a quarter-tone (50 cents) above note 65. According…
At this layer of note information, fact is the data will be either sent to a MIDI device or a plugin as a MIDI event, so you might as well keep it in the native format and not convert to a frequency.
If you want to make changes, you give ChatGPT the MP3 file and say "switch out the trumpet for a piano and make the singer a guy".