Launch HN: Traceloop (YC W23) – Detecting LLM Hallucinations with OpenTelemetry

Hey everyone, we are Nir and Gal from Traceloop (https://www.traceloop.com). We help teams understand when their LLM apps are failing or hallucinating at scale. See a demo: https://www.traceloop.com/video or try it yourself at https://www.traceloop.com/docs/demo.

When moving your LLM app to production, significant scale makes it harder for engineers and data scientists alike to understand when their LLM is hallucinating or returning malformed responses. When you get to millions of calls to OpenAI a month, methods like “LLM as a judge” can’t work at a reasonable cost or latency. So, what most people we talked to usually do is sample some generations by hand, maybe for some specific important customers, and manually look for errors or hallucinations.

Traceloop is a monitoring platform that detects when your LLM app fails. Under the hood, we built real-time versions of known metrics like faithfulness, relevancy, redundancy, and many others. These are loosely based on some well-known NLP metrics that work well for LLM-generated texts. We correlate them with changes we detect in your system - like updates to prompts or to the model you’re using - to detect regressions automatically.

Here are some cool examples we’ve seen with our customers -

1. Applying our QA relevancy metric to an entity extraction task, we managed to discover issues where the model was not extracting the right entities (like an address instead of a person’s name); or returning random answers like “I’m here! What can I help you with today?”.

2. Our soft-faithfulness metric was able to detect cases in summarization tasks where a model was completely making up stuff that never appeared in the original text.

One of the challenges we faced was figuring out how to collect the data that we need from our customers' LLM apps. That’s where OpenTelemetry came in handy. We built OpenLLMetry (https://github.com/traceloop/openllmetry), and announced it here almost a year ago. It standardized the use of OpenTelemetry to observe LLM apps. We realized that the concepts of traces, spans, metrics, and logs that were standardized with OpenTelemetry can easily extend to gen AI. We partnered with 20+ observability platforms to make sure that OpenLLMetry becomes the standard for GenAI observability and that the data that we collect can be sent to other platforms as well.

We plan to extend the metrics we provide to support agents that use tools, vision models, and other amazing developments in our fast-paced industry.

We invite you to give Traceloop a spin and are eager for your feedback! How do you track and debug hallucinations? How much has that been an issue for you? What types of hallucinations have you encountered?

Thanks!

We differentiate in 2 ways:

1. We focus on real-time monitoring. This is where we see the biggest pain with our customers, so we spent a lot of time researching and building the right metrics that can run at scale, fast and at low cost (and you can try them all in our platform).

2. OpenTelemetry - we think this is the best way to observe LLM app. It gives you a better understanding of how other parts of the system are interacting with your LLM. Say you're calling a vector DB, or making an HTTP call - you get them all on the same trace. It's also better for the customers - they're not vendor locked to us and can easily switch to another platform (or even use them in parallel).

congrats on launch!

the thing about OTel is that it is by nature vendor agnostic. so if i use OpenLLMetry, i should be able to pipe my otel traces to whatever existing o11y tool I use right? what is the benefit of a dedicated monitoring platform?

(not cynical, just inviting you to explain more)

Not OP here (but building in the same space). The reason you instrument LLM data is usually to improve quality/speed of your applications. The tools to extract the insights to enable that, and the integration with your LLM experimentation workflow is the differentiator between a general observability solution and LLM specific one.

oh cool. do you also consume OTel? or something else?