Live data from Hacker News

Show HN: Langfuse – Open-source observability and analytics for LLM apps

github.com

31–39 of 39 posts

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#31

Congrats on the launch! I have quite a few years of observability experience behind me and hand't really considered some of the unique aspects that LLMs bring into the picture. Here are a few thoughts, responses to your questions, and feedback items * Generally, I think you do a good job of having a clear, concise story and value proposition that is fairly early in a market where the number of people hitting these pr…

This reads like a book, thank you so much for putting this together! > About value prop: Thanks for the feedback! We are already trying to be as vocal about it as possible by writing great docs etc. but can probably do better. > PLG & OSS: thanks for the hint, we will be careful around managing deployments within customer VPCs. > Pricing: Currently picked storage as the first metric to price on as this varies a lot a…

Re: Supabase and timescaledb.

Just want to make a bit more clear, Supabase has the ability to do some distribution via replication, but it isn't a true multi-master DB.

Timescaledb does support a multi-node config (https://docs.timescale.com/self-hosted/latest/multinode-time...) on top of postgres but that isn't in the open-source apache-licensed version, instead it is only in Timescales's community BSL version which isn't license compatible with supabase

And yeah, please don't hesitate to reach out in regards to OTel... lots of opportunity but also not as simple ;)

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#32
Congrats on the release! Having built several LLM apps in the past months and embarking on a couple new ones, I’m excited to take a look at Langfuse.

Are there any alternatives you’d also suggest evaluating, and any particular strengths/weaknesses we should consider?

I’m also curious about doing quality metrics, benchmarking, regression testing, and skew measurement. I’ll dig further into Langfuse documentation (just watched the video so far) but I’d love any additional recommendations base on that.

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#33

Congrats on the launch! I have quite a few years of observability experience behind me and hand't really considered some of the unique aspects that LLMs bring into the picture. Here are a few thoughts, responses to your questions, and feedback items * Generally, I think you do a good job of having a clear, concise story and value proposition that is fairly early in a market where the number of people hitting these pr…

This reply is top notch. I agree with the points here. Definitely worth considering this advice.

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#34

Congrats on the release! I'm keenly interested in this space, as I believe that Observability is one of the top ways to steer LLMs to be more reliable in production. I noticed your SDKs use tracing concepts! Are there plans to implement OpenTelemetry support?

Shameless plug on https://proc.gg since you asked about OpenTelemetry support. The observability features are built upon Otel and I plan to open source it if there is considerable interests.

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#35
post #16

Earlier quoted context omitted.

None that I am aware of, and that is why I am very interested in learning about any red flags you might have found that motivated your decision to move off Postgres/Supabase and that others should be aware of.

I've been using it recently and I will say it is definitely harder to perform common time series queries than something like InfluxDB.

Can you elaborate? Harder as in writing queries or harder as in making queries performant?

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#37
post #16

Earlier quoted context omitted.

I've been using it recently and I will say it is definitely harder to perform common time series queries than something like InfluxDB.

Can you elaborate? Harder as in writing queries or harder as in making queries performant?

Writing them primarily, for example there isn't a super simple replacement for something like `derivative` (https://docs.influxdata.com/flux/v0.x/stdlib/universe/deriva...) in timescaledb.

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#38
Congrats on the launch! Sounds like an exciting project. Do you plan to store also the raw data (input + output)? It can be relevant for fine-tuning, optimizing costs, etc. Since you already store metadata, I think it makes sense to have a one-stop shop.

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#39
post #38

Congrats on the launch! Sounds like an exciting project. Do you plan to store also the raw data (input + output)? It can be relevant for fine-tuning, optimizing costs, etc. Since you already store metadata, I think it makes sense to have a one-stop shop.

Agree – Langfuse stores all prompts/completions, model configuration and metadata. Currently the GET API can be used to use the data for finetuning and we build a wrapper to access a filtered sample via the Python SDK.
Post reply on HN