Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Exception Monitoring and Response

githubengineering.com

Post reply on HN
Print view View original

11–20 of 22 posts

Re: Exception Monitoring and Response

#11

Post by zeeg »

jarus wrote: post #4

I can recommend Sentry which is open source and works great since many years! https://github.com/getsentry/sentry or hosted https://getsentry.com/

Always great to see the love for Sentry! We have definitely looked for inspiration from workflows in places like GitHub over the years. I recall when we first built Sentry and there wasn't really anything else like it out there. Probably around the same time Haystack was conceived, and the idea was starting to gain popularity in the Ruby community. It's amazing to see how much things have changed in the last almost-decade.

Re: Exception Monitoring and Response

#12

Post by zeeg »

bra-ket wrote: post #10

what are some Javascript libraries that allow creating such pretty dashboards?

My personal favorite is Highcharts, but it's a paid solution. Otherwise generally people build things on top of D3 these days. jQuery's Flot is also an easy entry tool for charts, but it's not as fancy out of the box.

Re: Exception Monitoring and Response

#14

Post by zeeg »

spydum wrote: post #2

well, disappoint.. show a shiny tool, then say how it took 6+ years to develop, and wont likely be open sourced. yes.. lot of us would like to implement something similar, just dont have the engineering resources to flush it out. if I had to take a stab at it, I'd say a good place to start is some ELK platform. Seems like it has almost all of the right components to build something like this. Just curious if they hav…

It's very hard to do this kind of thing by slurping logs (and is generally why ELK isn't considered a replacement for these tools). The first thing applications like Haystack try to solve is context. While I can't speak for Haystack (as I've never used it), Sentry using it's SDKs pulls in a ton of information that you would never get, or would at least be very hard to pull from logs. Examples of this include most of the HTTP request information, the authenticated user, context locals and surrounding code in the stacktrace, as well as any arbitrary context you might want to associate with it.

Re: Exception Monitoring and Response

#15

Post by gargarplex »

gargarplex wrote: post #9
jarus wrote: post #4

I can recommend Sentry which is open source and works great since many years! https://github.com/getsentry/sentry or hosted https://getsentry.com/

Lovely how the copy on their homepage is "Shit happens — Be on top of it." Trust me, that same organizational/cultural ethos applies to customer service, wherein I dealt with petulant, arrogant and occasionally wrong 20 somethings.

The support rep engineer was truculent. Petulant was the wrong word.

Re: Exception Monitoring and Response

#16

Post by RA_Fisher »

Rather than take a z-score, which is symmetrical, one idea for a small improvement might be to use a critical value from a chi-squared distribution. Since it's asymmetric, it'll probably fit the distribution better and offer a higher signal-to-noise ratio.

EDIT: or actually, since this is a count, might as well go straight for the poisson distribution or negative binomial -- same idea.

Re: Exception Monitoring and Response

#17

Post by fundead »

Looks like a top notch implementation that's had some serious effort put into it for solving their unique pain points around deploys and monitoring. My colleagues and I at Raygun (https://raygun.io) have created an error tracking service that from an overview ticks the boxes that Haystack supports, and a whole bunch more besides for dev teams who don't happen to be GitHub :)

Tools like these become pretty invaluable when releasing code into the wild, and aside from lifecycle benefits like greatly reducing the cost of the maintenance phase, an error tracker can also alert and hand the data needed to fix the bug right to the person who can, greatly increasing dev sanity (no more relying on support ticket ping pong, and removes the onus on the end user to report the bug). Also, surfacing the attack vectors and the frequency of which adversaries attack code exposed to the web is rather eye opening.

Re: Exception Monitoring and Response

#18

Post by mdeeks »

jarus wrote: post #4

I can recommend Sentry which is open source and works great since many years! https://github.com/getsentry/sentry or hosted https://getsentry.com/

Seconded! We use it extensively here. Really really valuable in our early days when we were first launching. As far as I'm concerned hooking into Sentry (or things like it) is a requirement for shipping and running a quality service. I even hook my cron jobs into it.

Re: Exception Monitoring and Response

#19

Post by joshdotsmith »

jarus wrote: post #4

I can recommend Sentry which is open source and works great since many years! https://github.com/getsentry/sentry or hosted https://getsentry.com/

We just interviewed Sentry founder David Cramer about exactly this topic: http://talkingcode.com/podcast/episode-10-david-cramer/
Post reply on HN