Viewing profile — ofermend

[dead]

[dead]

Given that coding agents like Claude Code or Codex are now quite good, there's of course a massive increase in PRs submitted to open source projects. Not all of them are great, som…

Gemini-3-flash is now on Vectara hallucination leaderboard, and rated at 13.5% grounded hallucination rate. https://github.com/vectara/hallucination-leaderboard

We just evaluated Nemotron-3 for Vectara's hallucination leaderboard. It scores at 9.6% hallucination rate, similar to qwen3-next-80b-a3b-thinking (9.3%) but of course it is much s…

GPT-5.2 just added to Vectara Hallucination Leaderboard. Definitely an improvement over GPT-5.1 - congrats to the team https://github.com/vectara/hallucination-leaderboard

Can't wait to try Opus 4.5 We just evaluated it for Vectara's grounded hallucination leaderboard: it scores at 10.9% hallucination rate, better than Gemini-3, GPT-5.1-high or Grok-…

If you have built AI agents in the last 6-12 months you know they fail a lot. I built this repository to be a community-curated list of failure modes, techniques to mitigate, and o…

Enterprise Deep Research is like "consumer" deep research just pointed at your private data, and I think may become the "killer app" of Agentic AI for business. Lots of valuable us…

One of the biggest challenges in RAG Evaluation is the assumption that you somehow can get the "source of truth" generated, specifically the set of "golden answers" (or golden chun…

Well, we expect AI to become AGI sometime in the future. Some say it's here, others say it's in 5 years or 50 years or whatever. So imagine AGI is here already (for sake of argumen…

We often debate "can humans trust AI?" An equally important question: can AI trust humans?

Shadow AI = Shadow IT 2.0 Large companies implementing generative AI are experiencing the re-emergence of all the issues and headaches we know well and remember, that were associat…

RAG Evaluation is difficult, primarily because it's hard to come up with "golden answers" (or golden chunks). We made Open-RAG-Eval to solve this - RAG Eval that only requires the …

A great day for open source, and so glad to see llama4 out. However, I'm a bit disappointed that the hallucination rates of Llama4 are not as low as I would have liked (TL;DR sligh…

This model is quite impressive. Not just useful for math/research with great reasoning, it also maintained a very low hallucination rate of 1.1% on Vectara Hallucination Leaderboar…

It is common these days to see in large companies multiple teams developing isolated RAG applications. This is similar to the problem of "Shadow IT" back in the early cloud era - c…