Is there a pros/cons comparison to Zipkin? Browsed the docs and couldn't find anything.
There is a section in the CNCF proposal:
https://github.com/cncf/toc/blob/master/proposals/jaeger.ado...
Jaeger – A Distributed Tracing System
31–40 of 66 posts
Re: Jaeger – A Distributed Tracing System
#32Earlier quoted context omitted.
Most folks will choose either Zipkin or Jaeger, but both are OpenTracing-compatible distributed tracing systems. You might find the Cloud Native Landscape useful for thinking about the options: https://github.com/cncf/landscape/blob/master/README.md Disclosure: I’m the executive director of CNCF, which just adopted Jaeger 2 weeks ago, and I’m an author of the landscape.
I have used that landscape document as a very informative point of reference for the last couple of months. Thank you for creating it.
You're very welcome. You cannot imagine the amount of criticism it has engendered.
Re: Jaeger – A Distributed Tracing System
#33Earlier quoted context omitted.
Most folks will choose either Zipkin or Jaeger, but both are OpenTracing-compatible distributed tracing systems. You might find the Cloud Native Landscape useful for thinking about the options: https://github.com/cncf/landscape/blob/master/README.md Disclosure: I’m the executive director of CNCF, which just adopted Jaeger 2 weeks ago, and I’m an author of the landscape.
Last I checked "OpenTracing-compatible" only went as far as using common terminology. Tbh I was a bit disappointed by this; has more been defined since? E.g. are there now shared schemas, APIs of sorts?
Yes, OpenTracing is an API, with bindings currently available in 9 languages. Please take a look.
Re: Jaeger – A Distributed Tracing System
#34Earlier quoted context omitted.
LightStep sounds interesting, but it seems to be closed source? That being the case, it's not hard to see why people are going with the (existing) OSS solutions. :/ UPDATE - Found this on GitHub, is this the whole thing? https://github.com/lightstep/lightstep-tracer-go If so, pointing people towards it from the .com website might help get people trying it out, as the .com website makes it seem non-OSS. :)
Lightstep is closed source. They have opensource libraries that get data into the system (like the go one you linked to) but to actually use it you need the hosted service. I hope to be releasing an open-source version of that approach in the next several months.
@necubi I do not disagree with you about the value of tail-based sampling. It has been an item on Jaeger roadmap from day one, but never of high enough priority to implement it. We're planning to do a POC by the end of this year. Jaeger's agent-based architecture was specifically designed to support that use case. If you're interested in collaborating on that, it would be great (https://github.com/uber/jaeger/issues/425).
Having said that, Dapper-style tracing systems with head-based sampling still provide enormous benefits that are often underestimated. In fact, even if we support tail-based sampling in the future, we're still going to run a certain portion of the traffic through probabilistic, head-based sampling, because it allows to reason about statistical patterns observed in the systems at large scale.
Re: Jaeger – A Distributed Tracing System
#35Does this run in production or is it used for testing? We wrote a distributed testing system ( https://github.com/gundb/panic-server ), so I'm trying to understand if integrating Jaeger would be helpful. If Jaeger is run on a production stack, I'd be curious to understand how that works (are there any tech talks on it yet?). If it is designed to run for tests, that makes sense, but then does it depend upon another di…
Yes, Jaeger is used in production at Uber and many other companies. Our original blog post on the history of Jaeger: https://eng.uber.com/distributed-tracing/
Re: Jaeger – A Distributed Tracing System
#36Is it something similar to sysdig?
Sysdig is focused on the networking layer. This is for the application layer (L7).
Re: Jaeger – A Distributed Tracing System
#37what is tracing?
In this context, it's the network-wide version of a stack trace. Instead of just method calls in a process, you take a top-level (HTTP) request and see all the various upstream service calls and their internal logic involved in completing that request. Useful for micro/multi-service based architectures but you can trace anything because it's a standard output format.
I have to say, you described tracing really well! I work on the OpenTracing project [1] (and at LightStep) and have been describing tracing to people for a while and your explanation is way better haha :-)
[1] opentracing.io
Re: Jaeger – A Distributed Tracing System
#38Earlier quoted context omitted.
That sounds pretty expensive. With dapper the trace span annotations do nothing if the request isn’t being traced. If it is being traced you might have significant costs, along the lines of sprintf(“%.03f”, ...) or other very cpu-intensive activity. This is OK when you trace one per million but when you trace everything you have to think about the cost. This could lead to either just using more CPU than you really wa…
Yeah, that's definitely an upside of the dapper approach--very, very minimal overhead for non-traced requests. However for the vast majority of use cases a bit of overhead (microseconds per span) tends to be unnoticeable, and the benefits in terms of introspectability are huge. In general, the overhead is mitigated by the fact that spans tend to be pretty large (on the ms-scale).
I guess it depends on your use case. If you are indeed interested in the 99p latency, then yes, the only way would be to trace everything. But in that case, couldn't you temporarily set the Dapper probability to 100%, record the data you need, and then turn it back down. It seems like a lot more malleable for different use cases.
Re: Jaeger – A Distributed Tracing System
#39Is there a pros/cons comparison to Zipkin? Browsed the docs and couldn't find anything.
100x better UI among other things. Jaeger has a dedicated team and resources and that shows. They are taking the time to write good docs, build demos, create a thoughtful UI, etc. I definitely recommend it. Being OpenTracing native also helps. Full disclosure, I work on the OpenTracing project.
Re: Jaeger – A Distributed Tracing System
#40Earlier quoted context omitted.
Lightstep is closed source. They have opensource libraries that get data into the system (like the go one you linked to) but to actually use it you need the hosted service. I hope to be releasing an open-source version of that approach in the next several months.
Thanks, please do. :)
100% agree with what necubi said. I work at LightStep and it's great to see OSS implementations coming out. The creators of LightStep actually are also the creators of Dapper haha.
LightStep specifically is meant for large-scale enterprise deployments and their specific needs and has focused on that for now.