Viewing profile — jneagu

We built HalluMix to evaluate how well hallucination detectors perform in the kinds of messy, high-stakes environments where LLMs are actually deployed: long-form outputs, multi-do…

Hey @echollama! Can you share more about what kind of integrations you are looking for? Also, when you say "enable customers to configure their usage" - what types of configuration…

"trained on licensed content" - this is a bit misleading. The correct framing is in the article: "made with content owner permission." Most (all?) LLMs out there are trained on lic…

I can save you a bit of market research and tell you that’s unfortunately not the case yet in the market today. There are a few reasons for it - the main one in my opinion being th…

The challenge with A/B experiments is how you design them to have sufficient power and draw a meaningful conclusion out of them. So, you either need a big % difference between the …

Nice! Overall, frameworks that re-prompt an LLM with feedback or failure modes of the original outputs do very well.

To your latter point - that’s where I think most of the value of LLMs in education is. They can explain code beyond the educational content that’s already available out there. They…

I am very curious to see how this is going to impact STEM education. Such a big part of an engineer's education happens informally by asking peers, teachers, and strangers question…

Anecdotally, synthetic data can get good if the generation involves a nugget of human labels/feedback that gets scaled up w/ a generative process.

Fair point - I actually had parsed OP's sentence differently. I'll edit my comment. I agree, LLMs performance for coding tasks is super biased in favor of well-represented language…

Yeah, There was a reference in a paywalled article a year ago ( https://www.theinformation.com/articles/openai-made-an-ai-br... ): "Sutskever's breakthrough allowed OpenAI to overc…

Edit: OP had actually qualified their statement to refer to only underrepresented coding languages. That's 100% true - LLM coding performance is super biased in favor of well-repre…

We've built SMELL (Subject-Matter Expert Language Liaison), a new framework that combines human expertise with LLMs to create feedback-informed, domain-specific LLM evaluators. One…

Hey HN, We are excited to reveal Quotient, a platform enabling developers to evaluate, improve and ship high-quality AI products through fast, real-world, data-backed experimentati…