Without benchmarking LLMs, you're likely overpaying
karllorey.com
Without benchmarking LLMs, you're likely overpaying
1–10 of 100 posts
Re: Without benchmarking LLMs, you're likely overpaying
#2> He's a non-technical founder building an AI-powered business.
It sounds like he's building some kind of ai support chat bot.
I despise these things.
Re: Without benchmarking LLMs, you're likely overpaying
#3I'd second this wholeheartedly
Since building a custom agent setup to replace copilot, adopting/adjusting Claude Code prompts, and giving it basic tools, gemini-3-flash is my go-to model unless I know it's a big and involved task. The model is really good at 1/10 the cost of pro, super fast by comparison, and some basic a/b testing shows little to no difference in output on the majority of tasks I used
Cut all my subs, spend less money, don't get rate limited
Re: Without benchmarking LLMs, you're likely overpaying
#4> He's a non-technical founder building an AI-powered business. It sounds like he's building some kind of ai support chat bot. I despise these things.
[flagged]
Re: Without benchmarking LLMs, you're likely overpaying
#5I'd second this wholeheartedly Since building a custom agent setup to replace copilot, adopting/adjusting Claude Code prompts, and giving it basic tools, gemini-3-flash is my go-to model unless I know it's a big and involved task. The model is really good at 1/10 the cost of pro, super fast by comparison, and some basic a/b testing shows little to no difference in output on the majority of tasks I used Cut all my sub…
Plus I've found that overall with "thinking" models, it's more like for memory, not even actual perf boost, it might even be worse because if it goes even slightly wrong on the "thinking" part, it'll then commit to that for the actual response
Re: Without benchmarking LLMs, you're likely overpaying
#6I'd second this wholeheartedly Since building a custom agent setup to replace copilot, adopting/adjusting Claude Code prompts, and giving it basic tools, gemini-3-flash is my go-to model unless I know it's a big and involved task. The model is really good at 1/10 the cost of pro, super fast by comparison, and some basic a/b testing shows little to no difference in output on the majority of tasks I used Cut all my sub…
[dead]
Re: Without benchmarking LLMs, you're likely overpaying
#7> He's a non-technical founder building an AI-powered business. It sounds like he's building some kind of ai support chat bot. I despise these things.
And the whole article is about promoting his benchmarking service, of course.
Re: Without benchmarking LLMs, you're likely overpaying
#8> He's a non-technical founder building an AI-powered business. It sounds like he's building some kind of ai support chat bot. I despise these things.
[flagged]
It's perfectly possible it's someone with deep domain experience, or someone who has product design or management skills. Regardless, dismissing these people out of pocket is not likely the best choice.
Re: Without benchmarking LLMs, you're likely overpaying
#9I'd second this wholeheartedly Since building a custom agent setup to replace copilot, adopting/adjusting Claude Code prompts, and giving it basic tools, gemini-3-flash is my go-to model unless I know it's a big and involved task. The model is really good at 1/10 the cost of pro, super fast by comparison, and some basic a/b testing shows little to no difference in output on the majority of tasks I used Cut all my sub…
Yeah, one of my first projects one of my buddies asked "Why aren't you using [ChatGPT 4.0] nano? It's 99% the effectiveness with 10% the price."
I've been using the smaller models ever since. Nano/mini, flash, etc.