Live data from Hacker News

Show HN: Langfuse – Open-source observability and analytics for LLM apps

github.com

11–20 of 39 posts

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#11

I’m curious if you investigated the TimescaleDB extension that is built into Supabase for your usecase? And if so, what was the pros and cons?

Thanks for the hint. Having only one fully managed db seems to be interesting to reduce needs for joining across databases and less operations overhead (managing uptime, setting up infra for testing, CI etc.) on our 2 people engineering "team". Timescale is definitely on the list to be at least an intermediary solution that would be faster to adopt than migrating to e.g. ClickHouse over time. As you mentioned it, any…

None that I am aware of, and that is why I am very interested in learning about any red flags you might have found that motivated your decision to move off Postgres/Supabase and that others should be aware of.

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#12

Earlier quoted context omitted.

Thanks for the hint. Having only one fully managed db seems to be interesting to reduce needs for joining across databases and less operations overhead (managing uptime, setting up infra for testing, CI etc.) on our 2 people engineering "team". Timescale is definitely on the list to be at least an intermediary solution that would be faster to adopt than migrating to e.g. ClickHouse over time. As you mentioned it, any…

None that I am aware of, and that is why I am very interested in learning about any red flags you might have found that motivated your decision to move off Postgres/Supabase and that others should be aware of.

Currently the project is all Postgres and the managed version is on Supabase. No scalability issues yet, we consider different OLAP options as we are mostly interested in faster analytical queries

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#13
Congrats on the release! I'm keenly interested in this space, as I believe that Observability is one of the top ways to steer LLMs to be more reliable in production.

I noticed your SDKs use tracing concepts! Are there plans to implement OpenTelemetry support?

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#16

Earlier quoted context omitted.

Thanks for the hint. Having only one fully managed db seems to be interesting to reduce needs for joining across databases and less operations overhead (managing uptime, setting up infra for testing, CI etc.) on our 2 people engineering "team". Timescale is definitely on the list to be at least an intermediary solution that would be faster to adopt than migrating to e.g. ClickHouse over time. As you mentioned it, any…

None that I am aware of, and that is why I am very interested in learning about any red flags you might have found that motivated your decision to move off Postgres/Supabase and that others should be aware of.

I've been using it recently and I will say it is definitely harder to perform common time series queries than something like InfluxDB.

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#18

Congrats on the release! I'm keenly interested in this space, as I believe that Observability is one of the top ways to steer LLMs to be more reliable in production. I noticed your SDKs use tracing concepts! Are there plans to implement OpenTelemetry support?

Thank you so much, fully share your sentiment on this and aligned our domain language to OpenTelemetry. Currently users add lots of metadata and configuration details to the trace by manually instrumenting it using the SDKs (or via Langchain integration). We are thinking about integrating OpenTelemetry, as this would be a step function on making integrations with apps easier. However, hadn't had the time yet to figure out how capture all the metadata that's relevant as context to the trace.

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#19
post #15

If you’re looking to replace Looker with open source and the ability to style it to your needs maybe a mix of cube.dev plus tremor.so would do the trick?

Thanks for the suggestion. We love tremor as it perfectly fits into our React/Tailwind setup. Cubeis great for collecting data from multiple resources, caching aggregates, and providing an API to call from our React FE. I think this could be a solution for the future in case we run into performance issues or end up having data stored in different databases. I am rather wondering how we can provide our users with a DD like dashboard experience. We would love to provide many different graphs, the ability to select and filter data, maybe even SQL like queries from the FE.

Re: Show HN: Langfuse – Open-source observability and analytics for LLM apps

#20
Congrats on the launch!

I have quite a few years of observability experience behind me and hand't really considered some of the unique aspects that LLMs bring into the picture. Here are a few thoughts, responses to your questions, and feedback items

* Generally, I think you do a good job of having a clear, concise story and value proposition that is fairly early in a market where the number of people hitting these problems is rapidly growing, which is a pretty nice place to be! But, I do think that can be a challenge in that you have to help people recognize the problem, which often means lots of content and lots of outreach.

* I think going open-source and following a PLG model of cloud/managed services is pretty reasonable way to go and certainly can be a leg up over the existing players, but I noticed in your pricing a note about enterprise support of self-hosting in customer VPC and dedicated instances. There is lots of money there... but it also can just be extremely big time sink for early stage teams, so I would be careful, or at least make sure you price it such that it supports hiring.

* Also on pricing, I wonder if doing this based on storage is how people would think about? Generally, I think about observability data in terms of events/sec first and then retention period. If you can make it work with a single usage based metric of storage, than that is great! but I would be concerned that 1) you aren't telling the user which plan can support throughput and 2) you could end up with some large variance in cost based on different usage patterns

* The biggest question I have is how much did you explore opentelemetry? Obviously, it is not as simple as just going and building your own API and SDK... but when I look at the capabilities, I could see opentelemetry being the underlying protocol with some thinner convenience wrappers on top. From your other comments, I understand that you see some ways in which this data is different than typical trace/observability data, but I do wonder if that choice will 1) scare off some companies that are already "all in" on otel and 2) you don't get any opportunity to use all of the stuff around otel, for example, Kafka integration if you someday need that.

* As far as your question about OLAP, I wouldn't rush it... In general, once you are big enough that the cost/scalability limitations of PG are looming, you will be a different company and know a lot more about the real requirements. I will also say that in all likelihood, ClickHouse is probably the right choice, but even knowing that, there are lots of different ways to tackle that problem (like using hosted vs self-managed) and the right way to do it will depend on usage patterns, cost structure, where you end up with enterprise dedicated / self-hosted, etc. I will mention though that timescaledb is not a bad way to maybe buy you a bit of headroom, but it is important to note that the timescaledb offered by supabase shouldn't be compared to timescaledb community / cloud. The supabase version isn't bad, it just isn't quite the same thing (i.e. no horizontal scalability)

Anyways, congrats again! It looks like you are off to a good start.

If you have any other questions for me, my email is in my profile.

Post reply on HN