Profiling Go Programs

#1

Post by alexandere » Wed, Nov 13, 2013, 4:53 PM UTC

Re: Profiling Go Programs

#2

Post by jamra » Wed, Nov 13, 2013, 5:27 PM UTC

I really like the usefulness of the pprof tool. I also like Rob Pike's book "The Unix Programming Environment". It's clear that Mr. Pike likes small command line programs that do an isolated task well.

I just wonder if it could be more beneficial to the community if Google invested in an IDE with a GUI for all of these tools.

Re: Profiling Go Programs

#3

Post by collinvandyck76 » Wed, Nov 13, 2013, 6:18 PM UTC

One thing I find frustrating about the net/http/pprof package is its init method, which does:

    http.Handle("/debug/pprof/", http.HandlerFunc(Index))
    http.Handle("/debug/pprof/cmdline", http.HandlerFunc(Cmdline))
    http.Handle("/debug/pprof/profile", http.HandlerFunc(Profile))
    http.Handle("/debug/pprof/symbol", http.HandlerFunc(Symbol))

In my particular case, I wanted all normal application traffic to be handled by my 8080 handler, and the /debug/* traffic handled by another listener on port 6060. Even if I explicitly link the above handler functions into my port 6060 ServeMux, because of the init() method it still registers across all of my handlers. I.e. port 8080 and 6060 both respond to, for example, /debug/pprof/cmdline.

Re: Profiling Go Programs

#4

Post by scarboy » Wed, Nov 13, 2013, 6:22 PM UTC

collinvandyck76 wrote: post #3

One thing I find frustrating about the net/http/pprof package is its init method, which does: http.Handle("/debug/pprof/", http.HandlerFunc(Index)) http.Handle("/debug/pprof/cmdline", http.HandlerFunc(Cmdline)) http.Handle("/debug/pprof/profile", http.HandlerFunc(Profile)) http.Handle("/debug/pprof/symbol", http.HandlerFunc(Symbol)) In my particular case, I wanted all normal application traffic to be handled by my 80…

net/http/pprof/pprof.go is just some boilerplate around the built in profiling functions. Copy it into your application and use your own init function.

Re: Profiling Go Programs

#5

Post by staunch » Wed, Nov 13, 2013, 6:34 PM UTC

Just recently felt the need to profile a tiny little Go program that I converted from Perl. Found out just how damn fast Perl's (C-based) regular expressions are. Go's Regexp library is absolutely awesome, but in this specific case significantly slower. Profiling was a piece of cake.

Re: Profiling Go Programs

#6

Post by stevvooe » Wed, Nov 13, 2013, 7:43 PM UTC

collinvandyck76 wrote: post #3

One thing I find frustrating about the net/http/pprof package is its init method, which does: http.Handle("/debug/pprof/", http.HandlerFunc(Index)) http.Handle("/debug/pprof/cmdline", http.HandlerFunc(Cmdline)) http.Handle("/debug/pprof/profile", http.HandlerFunc(Profile)) http.Handle("/debug/pprof/symbol", http.HandlerFunc(Symbol)) In my particular case, I wanted all normal application traffic to be handled by my 80…

Create a separate ServeMux for your application and let the utilities register to the DefaultServeMux via init (and you can even register your own!). Bind the DefaultServeMux to the debug port (6060) using http.ListenAndServe(debugPort, nil) and bind your app to its port (8080) with http.ListenAndServe(appPort, appMux).

Re: Profiling Go Programs

#7

Post by chimeracoder » Wed, Nov 13, 2013, 8:29 PM UTC

staunch wrote: post #5

Just recently felt the need to profile a tiny little Go program that I converted from Perl. Found out just how damn fast Perl's (C-based) regular expressions are. Go's Regexp library is absolutely awesome, but in this specific case significantly slower. Profiling was a piece of cake.

If you've already done the profiling, you're probably already aware of this, but for anybody else:

Go uses RE2, not PCRE (which Perl uses, obviously) for regular expressions: https://code.google.com/p/re2/. There is also a third-party PCRE library for Go somewhere, though the standard library uses RE2.

One of the advantages of RE2 is that it guarantees that the regular expression runs in linear time (in the length of the input), unlike backtracking-engines like PCRE (which are provably undecidable, like the halting problem).

However, aside from not supporting backtraces, it also can be slower on certain kinds of regexes and inputs.

As always, the best (only?) way to know which approach is better is to test (profile) - which, I agree, is a piece of cake in Go.

Re: Profiling Go Programs

#8

Post by johnarnow » Wed, Nov 13, 2013, 8:37 PM UTC

Does anyone have a link to download the slides? Slideshare apparently doesn't believe people can exist without either a Facebook or LinkedIn account or both.

Re: Profiling Go Programs

#9

Post by ushi » Wed, Nov 13, 2013, 9:00 PM UTC

For interested people: Here is the discussion about the mentioned pool package: https://code.google.com/p/go/issues/detail?id=4720

Re: Profiling Go Programs

#10

Post by collinvandyck76 » Wed, Nov 13, 2013, 9:58 PM UTC

stevvooe wrote: post #6

collinvandyck76 wrote: post #3

One thing I find frustrating about the net/http/pprof package is its init method, which does: http.Handle("/debug/pprof/", http.HandlerFunc(Index)) http.Handle("/debug/pprof/cmdline", http.HandlerFunc(Cmdline)) http.Handle("/debug/pprof/profile", http.HandlerFunc(Profile)) http.Handle("/debug/pprof/symbol", http.HandlerFunc(Symbol)) In my particular case, I wanted all normal application traffic to be handled by my 80…

Create a separate ServeMux for your application and let the utilities register to the DefaultServeMux via init (and you can even register your own!). Bind the DefaultServeMux to the debug port (6060) using http.ListenAndServe(debugPort, nil) and bind your app to its port (8080) with http.ListenAndServe(appPort, appMux).

This worked great, thanks.