The future of latency profiling in Go
rakyll.org
The future of latency profiling in Go
1–10 of 15 posts
Re: The future of latency profiling in Go
#2Using Opentracing and uber's Jaeger client is awesome. It's easy to implement, and Jaeger provides multiple different mechanisms for sampling traces (always on, guaranteed throughput etc.).
It's awesome to be able to see a UI giving you diagrams like this: https://www.dropbox.com/s/b2uhx77urpnuk9g/Screenshot%202017-... for your API.
Definitely would recommend - it beats old-school log tracing.
Re: The future of latency profiling in Go
#3Using Opentracing and uber's Jaeger client is awesome. It's easy to implement, and Jaeger provides multiple different mechanisms for sampling traces (always on, guaranteed throughput etc.). It's awesome to be able to see a UI giving you diagrams like this: https://www.dropbox.com/s/b2uhx77urpnuk9g/Screenshot%202017-... for your API. Definitely would recommend - it beats old-school log tracing.
Do I miss anything? OpenTracing mentions "A vendor-neutral open standard for distributed tracing." It reads like it's for implementors - is it useable stand alone + Jaeger as UI?
How can it be used - can you point me to a tutorial/blog post?
Re: The future of latency profiling in Go
#4Using Opentracing and uber's Jaeger client is awesome. It's easy to implement, and Jaeger provides multiple different mechanisms for sampling traces (always on, guaranteed throughput etc.). It's awesome to be able to see a UI giving you diagrams like this: https://www.dropbox.com/s/b2uhx77urpnuk9g/Screenshot%202017-... for your API. Definitely would recommend - it beats old-school log tracing.
Do I miss anything? OpenTracing mentions "A vendor-neutral open standard for distributed tracing." It reads like it's for implementors - is it useable stand alone + Jaeger as UI? How can it be used - can you point me to a tutorial/blog post?
Hey frik, former Uber engineer here. I never worked on Jaeger, but I loved using it at Uber.
If you want to get started, the documentation is at http://jaeger.readthedocs.io/en/latest/
We also have a blog post that describes the history of Jaegar and some of the design decisions. https://eng.uber.com/distributed-tracing/
Jaegar is compatible with the OpenTracing standard ( http://opentracing.io/documentation/pages/supported-tracers ) and is also compatible with the Zipkin backend and wire protocol ( http://zipkin.io/ ).
Re: The future of latency profiling in Go
#5Using Opentracing and uber's Jaeger client is awesome. It's easy to implement, and Jaeger provides multiple different mechanisms for sampling traces (always on, guaranteed throughput etc.). It's awesome to be able to see a UI giving you diagrams like this: https://www.dropbox.com/s/b2uhx77urpnuk9g/Screenshot%202017-... for your API. Definitely would recommend - it beats old-school log tracing.
Do I miss anything? OpenTracing mentions "A vendor-neutral open standard for distributed tracing." It reads like it's for implementors - is it useable stand alone + Jaeger as UI? How can it be used - can you point me to a tutorial/blog post?
Mm it's a little sparse right now - had to dig into the opentracing spec, dig in to the implementations and make it piecemeal.
OpenTracing is for everyone - both tracing implementors (eg. Jaeger engineers) and for you implementing tracing in your system. As an implementor in your system you mainly care about the "vocabulary" and "standards" opentracing defines. So, if you've got two functions (A which calls B) opentracing defines:
- Syntaxes to say that the span for "B" is a "childOf" "A". This gives you the graph I had above.
- Syntaxes to attach errors and other information to spans in a standard way (this lets any opentracing-compatible UI display/search errors etc): https://github.com/opentracing/specification/blob/master/sem...
OpenTracing also allows you to define functions that follow each other: https://github.com/opentracing/specification/blob/master/spe....
Plus, defines abstract ways of passing tracing contexts through service boundaries (ie. from client => API server => other microservices) via strings and HTTP headers.
Also, the best part of opentracing (in go) is that you can use it directly to set these fields (https://github.com/tonyhb/personal-starter/blob/master/pkg/a...). You can pull in the "github.com/opentracing/opentracing-go" library and use it to create and propagate your spans and contexts, and set errors using the default vocabulary (https://godoc.org/github.com/opentracing/opentracing-go/ext).
Been meaning to sort my blog out and write some posts on this as the implementation documentation is sparse. Some time soon.
Re: The future of latency profiling in Go
#6Earlier quoted context omitted.
Do I miss anything? OpenTracing mentions "A vendor-neutral open standard for distributed tracing." It reads like it's for implementors - is it useable stand alone + Jaeger as UI? How can it be used - can you point me to a tutorial/blog post?
Hey frik, former Uber engineer here. I never worked on Jaeger, but I loved using it at Uber. If you want to get started, the documentation is at http://jaeger.readthedocs.io/en/latest/ We also have a blog post that describes the history of Jaegar and some of the design decisions. https://eng.uber.com/distributed-tracing/ Jaegar is compatible with the OpenTracing standard ( http://opentracing.io/documentation/pages/su…
Thanks a lot, will try it out.
btw https://uber.github.io/jaeger/ still mentions "Backend components are implemented in Go (will be open sourced soon)" ...in the meantime the code go published as it seems: https://github.com/uber/jaeger
Re: The future of latency profiling in Go
#7For 1)., if you use a LLVM backed language, XRay supposedly should be it: https://llvm.org/docs/XRay.html
Re: The future of latency profiling in Go
#8PM for tracing at Google here.
As the blog post mentioned, we're collaborating with partners on a common RPC context proposal for tracing systems [1]. Some of the major contributors are currently on vacation, but there'll be be another round of comments and updates once they get back.
We're also working with the same partners on a project called Census - a set of tracing instrumentation libraries that all vendors can use and contribute to, though we're still in the very early days on this effort. We'll have more to announce later in the year (still need to publish additional libraries, set up a website, etc.), but you can follow the progress on GitHub here [2].
[1] https://github.com/TraceContext/tracecontext-spec/pull/1/fil... [2] https://github.com/census-instrumentation
Re: The future of latency profiling in Go
#9For 1)., if you use a LLVM backed language, XRay supposedly should be it: https://llvm.org/docs/XRay.html
LLVM is a future consideration for Go. One of the main reasons I don't want to tackle (1) right now if the possibility of benefiting from XRay even though it doesn't sort the case out for the vast majority: the gc users.
Re: The future of latency profiling in Go
#10PM for tracing at Google here. As the blog post mentioned, we're collaborating with partners on a common RPC context proposal for tracing systems [1]. Some of the major contributors are currently on vacation, but there'll be be another round of comments and updates once they get back. We're also working with the same partners on a project called Census - a set of tracing instrumentation libraries that all vendors can…
OMG, Census! It's one of the frameworks you miss the most once you no longer can use it. There is some preliminary version of it hidden deep inside gRPC, but it's great to hear that it's becoming a standalone project.