I don't mind at all.
The main benefit that I see (not being a linguist myself and seeing it second-hand, as it were) is saving time; it's quite a lot of effort for humans to listen to an audio recording and write it down in IPA. I didn't realize just how much effort was involved until I watched my wife researching her thesis, listening to the same recording over and over and over.
Also, there are possibilities for automated translation that the IPA transcription opens up. Speech-to-text models need different training than text-to-text translation models. The other day one of the linguists I know sent me a message, saying "I have a lot of recordings in language X" (I forget which language, but it doesn't really matter which language it is) "and I want to transcribe them. Is there a way to use AI to do that?" I thought it through, and realized that recording-to-IPA followed by IPA-to-correct-alphabet would be a way of dealing with the combinatorial explosion (according to https://www.ethnologue.com/insights/how-many-languages/ there are close to 7,200 languages spoken in the world, though "spoken" may be a misnomer since that number includes sign languages). There's no way anyone can train a model to convert about 7,000 languages from spoken form to written form; that's just way too many possibilities, and I suspect that audio models are necessarily large because audio is more complex information than written text. (If I'm wrong on that, someone please inform me). But although the human mouth (and throat) can produce dozens and dozens of different sounds, that's a finite number. (I think it approaches a couple hundred, but that's still a lot less than 7,000). So the number of distinct sounds and their IPA equivalents that the model would have to distinguish is smallish. Then another model could be trained on converting IPA to the alphabet (and spelling) of the world's 7,000-ish languages — and that would be a lot simpler than sound-to-spelling.
You're right that the recording is "more ideal" than any representation, and linguists will always archive the recording (in a lossless format, whenever possible, though if the original recording was done by a non-linguist who recorded it in MP3 then there's not much point in storing a WAV) alongside their transcription. This allows anyone else who might be doubtful on a point to go listen to the original recording and say "Yeah, Smith got the transcription wrong on that word. That's not an alveolar click, it's a velar click." But when you're writing a paper, you don't stick audio recordings into your document, you write down transcriptions. An audio recording without a transcription is not nearly as useful to linguists as having both. (EDIT to add: And such transcriptions, when you're writing papers about the sounds of a language, are pretty much always in IPA as well as in the language's standard alphabet. IPA isn't just a means to an end, it's also an end in itself. So even if a speech-to-text model existed for the language my linguist friend wanted to transcribe, he would still also want an IPA transcription as well, 90% of the time.)