Earlier quoted context omitted.
Are there any LLMs being widely used for audio classification? I know VLMs are being used a lot in image stuff. It always seems kind of silly to me to throw everything at an LLM. I know they’re huge and can automatically handle a huge number of tasks but something in me finds it wasteful when we could be creating easily trainable, cheap to run bespoke models for a lot of stuff
The harness that connects to a chatbot, API or voice interaction is the place to route requests to different systems. If you remember the early days of ChatGPT it explicitly said it was routing image generation to Dall-E after embellishing your request itself first. Determining which tool to use should be a lightweight operation but I’m not expert enough to understand exactly how much lighter than a full LLM call jus…
For instance, OCR is something that can be done locally with no access to a GPU but people (including me) still often use cloud hosted multi-modal large language models for it.