Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: You don't need to adopt new tools for LLM observability

github.com

Post reply on HN
Print view View original

21–24 of 24 posts

Re: Show HN: You don't need to adopt new tools for LLM observability

#21

Post by nirga »

epistasis wrote: post #20
nirga wrote: post #10

Earlier quoted context omitted.

Thanks for the issues - I'll fix it! :sweat_smile: Reg. Grafana and others - it's simple, just set the env vars - https://www.traceloop.com/docs/openllmetry/integrations/intr...

Thanks for effort of a reply, however I'm not quite willing to jump through hoops of creating accounts on two different services, plus the effort of integration, merely to see what the outputs might be. Shouldn't the benefit, the potential win to customers, be the selling point? Maybe others are willing to put in lots more effort than I am to see what the benefit is, but I've already spent a ton of time and am no clo…

You don't need to create an account in 2 services - you can just connect the SDK directly to Grafana - https://traceloop.com/docs/integration/grafana

Happy to assist if needed over slack - https://traceloop.com/slack

Re: Show HN: You don't need to adopt new tools for LLM observability

#22

Post by nirga »

Aqueous wrote: post #19
tracerbulletx wrote: post #14

Earlier quoted context omitted.

Pretty sure this just structures logs for requests to common 3rd party LLM providers. Which I guess is useful, but it's not some kind of problem unique to LLMs.

Correct- the summary is misleading marketing. This is just normal system / service observability. What people mean by observability in the LLM context is specific.

I wouldn't call it misleading marketing - it is what it is, similar to what you can get today from tools like Langsmith, etc - Observability for the LLM part of your system, but using your existing tools. You can further extend that to monitor specific LLM outputs - but that's just another layer on top of that.

Re: Show HN: You don't need to adopt new tools for LLM observability

#23

Post by Aqueous »

nirga wrote: post #22
Aqueous wrote: post #19

Earlier quoted context omitted.

Correct- the summary is misleading marketing. This is just normal system / service observability. What people mean by observability in the LLM context is specific.

I wouldn't call it misleading marketing - it is what it is, similar to what you can get today from tools like Langsmith, etc - Observability for the LLM part of your system, but using your existing tools. You can further extend that to monitor specific LLM outputs - but that's just another layer on top of that.

Not talking about just monitoring outputs though. I'm talking about monitoring the internals of the model as it reaches its output. The entire issue around interpretability / observability inside the LLM's model is the hard problem, one for which considerable resources are being dedicated to solve - not simply hooking the public-facing APIs up to observability tools like any other service API. This is just conventional telemetry. Calling this LLM observability implies there is something special about it and unique to LLMs in particular that enhances introspection into the AI model itself, which is not true. The title is highly misleading, classic startup-bro fake-it-til-you-make-it hustling crap, and deserves to be called out.

Re: Show HN: You don't need to adopt new tools for LLM observability

#24

Post by lmeyerov »

nirga wrote: post #7
lmeyerov wrote: post #6

Re:python, if we are already doing otel, how would this interop? Eg, if we don't want to break our current imports, and control where the new instrumentation goes (Fwiw, This is a great direction!)

Super easy - you can just use the standalone instrumentations directly - https://www.traceloop.com/docs/openllmetry/tracing/without-s...

... and we did, added to our Jaeger/Prometheus. Works great!
Post reply on HN