"every"
you're missing a lot
TTS: 11labs, PlayHT, Cartesia, iFLYTEK, AWS Polly, Deepgram Aura
STT: Deepgram (multiple models, including Whisper), Gladia Whisper, Soniox
just off the top of my head (it's my dayjob!)
Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
51–60 of 82 posts
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#52These are hard to keep updated. I find they usually fall off. It would be cool to have one, but honestly, this one already doesn't even have 4o and pro on it which if it was being maintained, it obviously would. Updating a table shouldn't take days. It's like a one minute event.
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#53Great! I wish there was a "bang to buck" value. Some way to know the cheapest model I could use for creating structured data from unstructured text, reliably. Using gpt4o-mini which is cheap but wouldn't know if anything cheaper could do the job too.
I always plug openrouter.ai for making cross-model comparisons. It's my general goto for random stuff. (I am not affiliated, just a user)
I love the idea of openrouter. I hadn't realized until recently though that you don't necessarily know what quantization a certain provider is running. And of course context size can vary widely from provider to provider for the same model. This blog post had great food for thought https://aider.chat/2024/11/21/quantization.html
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#54OP, were you inspired by this LLM comparison tool? https://whatllm.vercel.app The tables are very similar - though you've added a custom calculator which is a nice touch. Also for the Versus Comparison, it might be nice to have a checkbox that when clicked highlights the superlative fields of each LLM at a glance.
Data in this tool is from https://artificialanalysis.ai/ on October 13 2024 and so is a little of out date.
This page has up to date information of all models and providers: https://artificialanalysis.ai/leaderboards/providers We also on other pages cover Speech to Text, Text to Speech, Text to Image, Text to Video.
Note I'm one of the creators of Artificial Analysis.
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#55I was surprised: what is that the model that costs the most per token? Luminous-Supreme-Control
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#56Love the UI and table layout. Have you though about showing the different VRAM requirements for models?
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#57[deleted]
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#58I like the idea of more comparisons of models. Are there plans to add independent analyses of these models or is it only an aggregation of input limits? How do you see this differing from or adding to other analyses such as: https://artificialanalysis.ai https://huggingface.co/spaces/TTS-AGI/TTS-Arena https://huggingface.co/spaces/hf-audio/open_asr_leaderboard https://huggingface.co/spaces/TIGER-Lab/GenAI-Arena Great…
the gradio ui looks ugly imo, that's why I used shadcn and next.js to make the website look good. I'll try to make it as user-friendly as possible. Most of the websites are ugly + too technical.
I want to point out you dodged the data question, and there's a reason for it.
I like your work visually on first glance, god knows you're right about gradio, even if its irrelevant.
But peddling extremely limited, out of date, versions of other people's data, trumps that, especially with this tagline. "A website to compare every AI model: LLMs, TTSs, STTs"
It is a handful of LLMs, then one TTS model, then one STT model, both with 0 data. And it's worth pointing out, since this endeavor is motivated by design trumping all: all the columns are for LLM data.
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#59One thing that stands out playing with the sorting is that Google's Gemini claims to have a context window more than 10x that of most of its competition. Has anyone experimented with this to see if its useful context window is actually anything close to that?
In my own experiments with the chat models they seem to lose the plot after about 10 replies unless constantly "refreshed", which is a tiny fraction of the supposed 128000 token input length that 4o has. Does Gemini actually do something dramatically differently, or is their 3 million token context window pure marketing nonsense?
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#60I feel like the number is still a bit lacking, especially since many models made by Chinese companies are not represented, like speech-to-text.
As far as I know, there's a volcano engine in China that has impressive text-to-speech capabilities. Many local companies are using this model.