great to see how you guys worked with the community on discord over the last year to build Langfuse
Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps
51–60 of 64 posts
Re: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps
#52great to see how you guys worked with the community on discord over the last year to build Langfuse
Thanks! IMO, Discord is good, but GitHub Discussions is the better option for building a growing open-source community. It is indexed and makes it easier to revisit conversations weeks later. Currently we use both but have a strong preference for GitHub Discussions.
Re: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps
#53Earlier quoted context omitted.
Bunch of them : Langsmith, Lunary, Phoenix Arize, Portkey, Datadog and Helicone. We also picked Langfuse - more details here: https://www.nonbios.ai/post/the-nonbios-llm-observability-pi...
Thanks, this post was insightful. I laughed at the reason why you rejected Arize Phoenix, I had similar thoughts while going through their site!=) > "Another notable feature of Langfuse is the use of a model as a judge ... this is not enabled in the free version/self-hosted version" I think you can add LLM-as-judge to the self-hosted version of Langfuse by defining your own evaluation pipeline: https://langfuse.com/d…
Thanks for the pointer !
We are actually toying with building out a prompt evaluation platform and were considering extending langfuse. Maybe just use this instead.
Re: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps
#54Thread is filled with positive reviews.. Little odd
Re: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps
#55Earlier quoted context omitted.
Bunch of them : Langsmith, Lunary, Phoenix Arize, Portkey, Datadog and Helicone. We also picked Langfuse - more details here: https://www.nonbios.ai/post/the-nonbios-llm-observability-pi...
Thanks for sharing your blogpost. We had a similar journey. I installed and tried both Langfuse and Phoenix and ended up choosing Langfuse due to some versioning conflicts on the python dependency. I’m curious if your thoughts change after V3? I also liked that it only depended on Postgres but the scalable version requires other dependencies. The thing I liked about Phoenix is that it uses OpenTelemetry. In the end w…
So we are still on V2.7 - works pretty good for us. Havent tried V3 yet, and not looking to upgrade. I think the next big feature set we are looking for is a prompt evaluation system.
But we are coming around to the view that it is a big enough problem to have dedicated saas, rather than piggy back on observability saas. At NonBioS, we have very complex requirements - so we might just end up building it up from the ground up.
Re: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps
#56Thread is filled with positive reviews.. Little odd
> Make sure your friends don't post booster comments. That's not allowed on HN. Our readers have a nose for this, and will sniff them out and flame you. That will damage your reputation—and ours—and we may have to bury your thread.
Re: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps
#57Earlier quoted context omitted.
What are other potential platforms?
Bunch of them : Langsmith, Lunary, Phoenix Arize, Portkey, Datadog and Helicone. We also picked Langfuse - more details here: https://www.nonbios.ai/post/the-nonbios-llm-observability-pi...
"Langsmith appeared popular, but we had encountered challenges with Langchain from the same company, finding it overly complex for previous NonBioS tooling. We rewrote our systems to remove dependencies on Langchain and chose not to proceed with Langsmith as it seemed strongly coupled with Langchain."
I've never really used Langchain, but setup Langsmith with my own project quite quickly. It's very similar to setting up Langfuse, activated with a wrapper around the OpenAI library. (Though I haven't looked into the metadata and tracing yet.)
Functionally the two seem very similar. I'm looking at both and am having a hard time figuring out differences.
Re: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps
#58Awesome improvements. How does this compare to Braintrust? I've played with it a bit and we're gearing up to implement a solution in during the Christmas lull.
We use various LLMs as a core part of our app but I'm looking for ways to more quickly iterate on our prompts, test different LLM outputs against each other, etc. ideally while minimizing deploys. Would Langfuse serve that purpose?
Re: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps
#59(unsolicited review) we've been happy adopters of LangFuse at AINews ( https://smol.ai/news ). ive been tracking the llm ops landscape ( https://www.latent.space/p/braintrust ) for a while and its very nice to have an open source solution that is so comprehensive and intuitive! reflections/thoughts on where this field goes next: 1. i wonder if there are new ops solutions for the realtime apis popping up 2. retries fo…
appreciate your constructive feedback! > i wonder if there are new ops solutions for the realtime apis popping up This is something we have spent quite some time on already, both on designs internally and talking to teams using Langfuse with realtime applications. IMO the usage patterns are still developing and the data capturing/visualization needs across teams is not aligned. What matters: (1) capture streams, (2)…
im too lazy to send traces for now haha. maybe in future when it REALLY bothers me.
good luck keep going.
Re: Launch HN: Langfuse (YC W23) – OSS Tracing and Workflows to Improve LLM Apps
#60Are the Traces OpenTelemetry Compatible?