Great
About AI Evals
41–46 of 46 posts
Re: About AI Evals
#42People should be demanding consistency and traceability from the model vendors checked by some tool perhaps like this. This may tell you when the vendor changed something but there is otherwise no recourse?
Agreed! FWIW I am attempting to create an open-source wiki/watchdog eval platform -- weval.org -- , so we can all keep an eye on LLMs, their biases, and their general competencies without relyong in the AI providers marking their own homework. I really believe this needs to exist to express our needs and hold model creators to account. Especially as model drift and manipulation becomes a risk.
Re: About AI Evals
#43This reads like a collection of ad hoc advice overfitted to experience that is probably obsolete or will be tomorrow. And we don’t even know if it does fit the author’s experience. I am looking for solid evidence of the efficacy of folk theories about how to make AI perform evaluation. Seems to me a bunch of people are hoping that AI can test AI, and that it can to some degree. But in the end AI cannot be accountable…
Hamel wrote a whole lot more about the "LLM as a judge" pattern (where you use LLMs to evaluate the output of other LLMs) here: https://hamel.dev/blog/posts/llm-judge/
I really recommend people study the measurement frailties and prompting sensitivities of LLM judges before employing them. They're valuable, but should be used with complete understanding of the risks: https://www.cip.org/blog/llm-judges-are-unreliable
Re: About AI Evals
#44I'm biased in that I work on an open source project in this space, but I would strongly recommend starting with a free/open source platform for debugging/tracing, annotating, and building custom evals. This niche of the field has come a very long way just over the last 12 months, and the tooling is so much better than it used to be. Trying to do this from scratch, beyond a "kinda sorta good enough for now" project, i…
Yes I'm not sure I really want to vibe code something that does auto evals on a sample of my OTEL traces any more than I want to build my own analytics library. Alternatives to Opik include Braintrust (closed), Promptfoo (open, https://github.com/promptfoo/promptfoo ) and Laminar (open, https://github.com/lmnr-ai/lmnr ).
I've used Promptfoo for client projects and really enjoyed it once I got my head around it.
Re: About AI Evals
#45Some great info, but I have to disagree with this: > Q: How much time should I spend on model selection? > Many developers fixate on model selection as the primary way to improve their LLM applications. Start with error analysis to understand your failure modes before considering model switching. As Hamel noted in office hours, “I suggest not thinking of switching model as the main axes of how to improve your system…
I think the key part if that advice is the without evidence bit: > I suggest not thinking of switching model as the main axes of how to improve your system off the bat without evidence. If you try to fix problems by switching from eg Gemini 2.5 Flash to OpenAI o3 but you don't have any evals in place how will you tell if the model switch actually helped?
This is not ideal, but it's pragmatic - or at least was, for the last two years - since new models showed large improvements across the board. If your main problem was capability, not cost, then switching was an easy win - from GPT-3.5 to GPT-4, from GPT-4 to say Sonnet 3.5, to Gemini 2.5 Pro, now to Opus (if you can afford it); or from Sonnet 3.5 to Deepseek-R1, to o3 (and that doesn't even consider multi-model solutions). The jump in capability was usually quite apparent.
Of course, Hamel is right too. In the long run, people will need to take more scientific approach. They already do, if inference costs are the main concern.
Re: About AI Evals
#46One of the biggest challenges in RAG Evaluation is the assumption that you somehow can get the "source of truth" generated, specifically the set of "golden answers" (or golden chunks/documents).
In practice that is extremely difficult and non scalable.
Open-RAG-Eval is a new open source project that aims to address that via reference-free evaluation such as UMBRELA and AutoNuggetizer scores.
Repo: https://github.com/vectara/open-rag-eval and a nice UI to use this with: openevaluation.ai
Would love to hear feedback on this after you try it out and what you might want to see on the roadmap.