[dead]
Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
61–70 of 82 posts
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#62One thing that stands out playing with the sorting is that Google's Gemini claims to have a context window more than 10x that of most of its competition. Has anyone experimented with this to see if its useful context window is actually anything close to that? In my own experiments with the chat models they seem to lose the plot after about 10 replies unless constantly "refreshed", which is a tiny fraction of the supp…
What tactics do you use to refresh while using them?
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#63One thing that stands out playing with the sorting is that Google's Gemini claims to have a context window more than 10x that of most of its competition. Has anyone experimented with this to see if its useful context window is actually anything close to that? In my own experiments with the chat models they seem to lose the plot after about 10 replies unless constantly "refreshed", which is a tiny fraction of the supp…
What tactics do you use to refresh while using them?
I tend to use a sentence along these lines:
"Give me a straightforward summary of what we discussed so far, someone who didn't read the above should understand the details. Don't be too verbose."
Then i just continue from there or simply use this as a seed in another fresh chat.
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#64One thing that stands out playing with the sorting is that Google's Gemini claims to have a context window more than 10x that of most of its competition. Has anyone experimented with this to see if its useful context window is actually anything close to that? In my own experiments with the chat models they seem to lose the plot after about 10 replies unless constantly "refreshed", which is a tiny fraction of the supp…
What tactics do you use to refresh while using them?
I don't have a strategy that I like—it just amounts to having to say "you forgot about requirement X, try again keeping that in mind".
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#65One thing that stands out playing with the sorting is that Google's Gemini claims to have a context window more than 10x that of most of its competition. Has anyone experimented with this to see if its useful context window is actually anything close to that? In my own experiments with the chat models they seem to lose the plot after about 10 replies unless constantly "refreshed", which is a tiny fraction of the supp…
When the released it they specifically focused on the accurate recall across the context window. There are a bunch of demos of things like giving it a whole movie as input (frame every N seconds plus script or something) and asking for highly specific facts).
Anecdotally, I use NotebookLM a bit, and while that’s probably RAG plus large contexts (to be clear, this is a guess not based on inside knowledge), it seems very accurate.
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#66One thing that stands out playing with the sorting is that Google's Gemini claims to have a context window more than 10x that of most of its competition. Has anyone experimented with this to see if its useful context window is actually anything close to that? In my own experiments with the chat models they seem to lose the plot after about 10 replies unless constantly "refreshed", which is a tiny fraction of the supp…
https://github.com/NVIDIA/RULER results in benchmark other than needle in haystack seem solid all the way to 128k
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#67Earlier quoted context omitted.
I always plug openrouter.ai for making cross-model comparisons. It's my general goto for random stuff. (I am not affiliated, just a user)
I love the idea of openrouter. I hadn't realized until recently though that you don't necessarily know what quantization a certain provider is running. And of course context size can vary widely from provider to provider for the same model. This blog post had great food for thought https://aider.chat/2024/11/21/quantization.html
To expand a little, some providers may apply more aggressive optimization in periods of high load.
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#68One thing that stands out playing with the sorting is that Google's Gemini claims to have a context window more than 10x that of most of its competition. Has anyone experimented with this to see if its useful context window is actually anything close to that? In my own experiments with the chat models they seem to lose the plot after about 10 replies unless constantly "refreshed", which is a tiny fraction of the supp…
https://github.com/NVIDIA/RULER results in benchmark other than needle in haystack seem solid all the way to 128k
Thanks, this is exactly the kind of info I was hoping existed.
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#69Logs emitted during the build, or test results, or metrics captured during the build (such as how long it took)... these can all themselves be build outputs.
I've got one where "deploying" means updating a few version strings and image reverences in a different repo. The "build" clones that repo and makes the changes in the necessary spots and makes a commit. Yes, the side effect I want is that the commit gets pushed--which requires my ssh key which is not a build input--but I sort of prefer doing that bit by hand.
Re: Show HN: Countless.dev – A website to compare every AI model: LLMs, TTSs, STTs
#70you guys might also like http://llmprices.dev, similar but it's automatically updated with the latest info every 24h