Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs

countless.dev

Post reply on HN
Print view View original

41–50 of 82 posts

Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs

#42

Post by shepherdjerred »

gtirloni wrote: post #29

Tangent question: is there anything better on the desktop than ChatGPT's native client? I find it too simple to organize chats but I'm having a hard time evaluating the dozen or so apps (most are disguise for some company's API service). Any recommendations? macOS/Linux compatibility preferred.

I've liked Machato: https://untimelyunicorn.gumroad.com/l/machato

Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs

#45

Post by botro »

ursaguild wrote: post #10

I like the idea of more comparisons of models. Are there plans to add independent analyses of these models or is it only an aggregation of input limits? How do you see this differing from or adding to other analyses such as: https://artificialanalysis.ai https://huggingface.co/spaces/TTS-AGI/TTS-Arena https://huggingface.co/spaces/hf-audio/open_asr_leaderboard https://huggingface.co/spaces/TIGER-Lab/GenAI-Arena Great…

I made https://aimodelreview.com/ to compare the outputs of LLMs over a variety of prompts and categories, allowing a side by side comparison between them. I ran each prompt 4 times for different temperature values and that's available as a toggle.

I was going to add reviews on each model but ran out of steam. Some users have messaged me saying the comparisons are still helpful to them in getting a sense of how different models respond to the same prompt and how temperature affects the same models output on the same prompt.

Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs

#46

Post by xnx »

vunderba wrote: post #32

OP, were you inspired by this LLM comparison tool? https://whatllm.vercel.app The tables are very similar - though you've added a custom calculator which is a nice touch. Also for the Versus Comparison, it might be nice to have a checkbox that when clicked highlights the superlative fields of each LLM at a glance.

Thanks for sharing. That's a better tool.

Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs

#47

Post by sdesol »

karpatic wrote: post #27

Great! I wish there was a "bang to buck" value. Some way to know the cheapest model I could use for creating structured data from unstructured text, reliably. Using gpt4o-mini which is cheap but wouldn't know if anything cheaper could do the job too.

I haven't found a model at the price point of GPT-4o mini that is as capable. Based on the hype surrounding Llama 3.3 70B, it might be that one though. On Deepinfra, input tokens are more expensive, but the output token is cheaper so I would say they are probably equivalent in price.

Also, best bang for the buck is very subjective, since one person might need it to work for one use case vs somebody else, who needs it for more.

Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs

#48

Post by nostrebored »

jampa wrote: post #30
karpatic wrote: post #27

Great! I wish there was a "bang to buck" value. Some way to know the cheapest model I could use for creating structured data from unstructured text, reliably. Using gpt4o-mini which is cheap but wouldn't know if anything cheaper could do the job too.

Take a look at Gemini Flash 1.5. I had videos I needed to turn into structured notes, and the result was satisfactory (even better than the Gemini 1.5 Pro, for some reason). https://jampauchoa.substack.com/i/151329856/ai-studio . According to this website, the cost is half of the gpt4-o mini. 0.15 vs 0.07 per 1M token.

Seconding Gemini flash for structured outputs. Have had some quite large jobs I’ve been happy with.

Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs

#50

Post by andrewmcwatters »

xnx wrote: post #46
vunderba wrote: post #32

OP, were you inspired by this LLM comparison tool? https://whatllm.vercel.app The tables are very similar - though you've added a custom calculator which is a nice touch. Also for the Versus Comparison, it might be nice to have a checkbox that when clicked highlights the superlative fields of each LLM at a glance.

Thanks for sharing. That's a better tool.

Both seem to have great value. Some information is missing from Vercel's tables.
Post reply on HN