Introducing ChatGPT and Whisper APIs

I expect some level of caching and even request bucketing by similarity is possible.

How many users come with the same prompt?

This is advanced stuff. Thank you for writing it down.

The same things were said during the industrial revolution, or the internet revolution. Guess what happened? Labor market adapted.

Transcript of 20230302 170602.m4a (at 2023-03-02 1711.02).txt

   Hey friend, your use case sounds really interesting.
   Actually that's why I created this app initially.
   I really love riding around on my bike in the city
   and doing voice memo debriefs about whatever.
   I also like to do it walking around outside.
   And as you say, the trouble with that is wind distortion.
   Full stop.
   On a day where it's not too windy, it's not too bad.
   These models can totally pull the text from it.
   But the more distortion you have,
   the more of a disaster it is.
   And I don't know anything about the multilingual case,
   but for English, I definitely find that
   small is more than enough if you have good quality audio.
   Medium, you might wanna use it
   if there's some kind of distortion
   that's causing errors in the small.
   But if you have really good quality audio,
   even tiny is enough.
   I mean, it won't get some sort of rare words.
   So small is basically good enough for English anyway.
   Aligning with what you said,
   I remember seeing in the whisper paper
   that the performance actually decreases,
   the word error rate increases from the medium
   to the large model in the multilingual case,
   which is kind of interesting.
   So basically medium, I think, is all you really need.
   I think doing large, running large locally
   is probably a waste of time.
   But this doesn't apply to the OpenAI API case
   because they're running their own sort of special model.
   It's very fast.
   Plus they're kind of going to be retraining it
   so continuing to improve it over time.
   So obviously there's that, which is cool.
   I think basically I did extensive research
   and experiment with this,
   with trying to clean up audio for the transcription.
   And there's basically no way to do it.
   Like if you have a medium to bad level of noise
   that the transcription models can still work with,
   you're fine.
   Just go with that.
   But in that case,
   there's no point actually trying to denoise the sound first.
   That just seems to reduce the signal
   and it actually increases the word error rate.
   So just give them the raw distorted, windy audio
   and the models will do the best they can.
   You can't actually improve it, I found.
   I tried all kinds of different ways to process it
   and none of it actually improved it,
   including like the best possible denoiser I could find,
   which is the Facebook research denoiser.
   So my conclusion was that, okay,
   I found a sort of a fundamental physical limit
   and I think using denoising is really only good for humans.
   Like if you want to listen to the audio again,
   you don't want to hear all that wind probably.
   And for medium to bad,
   but not extreme levels of wind distortion
   or other kinds of noise distortion,
   you can use a denoiser like the Facebook research one
   and that will totally or nearly totally
   kind of reduce all that noise.
   But I basically decided that the only way
   to kind of get better quality audio
   or to get better quality transcripts,
   if you're doing it outside on a windy day,
   is not to go with software enhancement
   because it doesn't do anything, it doesn't achieve anything.
   I tried everything possible
   and nothing produced results in the extreme distortion case.
   So what I decided is that's basically a limit,
   physical limit and so the best way to do it I think
   is to change your microphone setup,
   have some sort of baffle around it,
   maybe one of those little microphones
   that can clip close to your throat or,
   I don't know, I'm no expert,
   but I think basically you can find a way
   to get better quality, less distorted audio
   outside by changing the microphone setup,
   adding one of those wind baffles or something.
   And I think that's basically all you can do essentially.
   And then once you have reduced
   a lot of that noise distortion,
   even if the resulting audio is still distorted,
   if it's not too bad, the models can be okay with it.

End of Transcript (created with WisprNote)

Obviously, that's business 101. Consumers should consider that ultimately all these cheap too-good-to-be-true offers cost them more than if they initially paid a bit more, but had more long term competition in the market. Amazon was the same way, they lost money for years but now have a quasi monopoly in many countries. There's a general trend towards such ventures supported by backers with deep pockets. And so the few extremely wealthy people get richer and richer.