Observability for LLM apps with structlog and DuckDB

Post by verdverm » Fri, Jun 07, 2024, 2:22 PM UTC

pranay01 wrote: post #10

verdverm wrote: post #4

This seems limited compared to the observability frameworks that integrate with the popular LLM frameworks like LangChain and LlamaIndex. When you use those, you want automatic traces for the nested calls hidden down the function call chain. This really seems more like a Flask API tracing example with as LLM call as an endpoint. Dare I call it a shallow developer marketing piece? re: LLM observability more generally,…

If you are looking for ways to monitor LLMs which tie in with the rest of your observability stack, check out these docs [1] on monitoring LLMs with SigNoz. The tutorial with OpenLLMetry uses Otel format - [1] https://signoz.io/docs/community/llm-monitoring/