Live data from Hacker News

Sentry.io outage

status.sentry.io

51–60 of 84 posts

Re: Sentry.io outage

#51

Earlier quoted context omitted.

Plus, it's literally the first time I've seen them having an outage over the years so it seems like they're generally doing well on staying up.

I think this was their Big One: https://blog.sentry.io/2015/07/23/transaction-id-wraparound-...

... from 7 years ago.

They've been pretty solid.

I've been really happy with Sentry. No muss, no fuss as a vendor.

I wish they would buy/bundle a status monitoring tool.

Re: Sentry.io outage

#52

Earlier quoted context omitted.

Yes the CI should fail, or else you would be deploying something that can't match your errors to your source code (via the sourcemaps which is what presumably gets uploaded to sentry on the CI run).

Take in advance that I don't know Sentry, nor webpack et al but, in my mind, I'd like to have an opportunity to continue my build process with a big, red warning. I prefer a 98% accurate error reporting better than a 100% one I can't push to production. Or maybe I'm not understanding well the value Sentry offers, of course.

It can be desired to continue on failure, the tradeoff is that there is then a release out there with lacking error-sourcecode mapping.

I think if your release cycle is longer, then you will prefer not to deploy without sourcemaps.

Re: Sentry.io outage

#53
post #47
post #21

Earlier quoted context omitted.

I am using it, but it is not fun to use. It spawns 28 Docker containers (yes, that's the actual number), including things like nginx, postgres, memcached, redis, kafka, zookeeper, clickhouse. I've tried upgrading it once and it failed miserably, after which I simply started from scratch. I would like to use their hosted version, or a more stable paid on-prem version. Pricing is not the issue for me, but I don't want…

Would you mind sharing which other saas products you are so posting?

Well the bare minimum that you have to tell your customers is your cloud provider / hosting provider, then you sometimes have a separate provider for Object Storage (S3 or Backblaze), maybe another one for email delivery (Mailgun). You might have an invoicing/CRM portal (Stripe), SCM (Github), Chat (Slack), your own email setup (Gmail). And that is not even talking about any client-side tracking, things like Google Analytics etc. So it really makes sense to cut down the number of third parties, especially if there is any scenario in which they would be handling PII, and especially if you're selling to customers in heavily regulated industries.

Re: Sentry.io outage

#54
post #7
post #6

Sentry is a great product, I love even more that you can self-host it; otherwise I would be one of the people denigrating everyone for running their entire operation on SaaS products. As it stands if their reliability leaves something to be desired: run your own. These things happen, they’re more common the more complicated or huge your setup. Good luck to them in getting it resolved.

Has anyone here used their self hosted version? I always assume that running an on-prem version of a SAAS offering is going to be a shit show, but I'm curious if anyone here uses it?

I've being running their self-hosted version, and it's kind of OK.

Nowhere near as nice as, say, GitLab's Ubuntu packages which you can simply apt-get install, unattended automatic (security) updates are available and work well, and it's been mostly a set-up-and-forget type of thing. For GitLab, backups and most ops procedures are well documented.

Sentry, on the other hand, requires you to run a script to upgrade with their docker-compose thingies. You must remember to upgrade.

Self-hosted Sentry runs on docker, which on Ubuntu is a bit of a security shitshow what with docker side-stepping UFW and all that.

Recently Sentry tripled (seems like) the number of containers they spin up, and it seems much heavier than a couple of years ago. You can still get away with 8G RAM, so it's not really that bad, to be honest.

Backing up Sentry was simple enough. However, a lot of questions are answered in the style "if you want to know more, buy the enterprise version". But, I've backed up and restored a Sentry system and it worked out OK, so I guess we're fine. Unless they change something, and then we're hosed without realizing it.

So, Sentry's self-hosted version is OK, but clearly not their number one priority. I have no idea what Sentry would look like with a self-hosted distributed/HA setup. Our data protection policies would absolutely prohibit us from using their cloud services, so maybe I'll have find out, but I'm not looking forward to that.

Re: Sentry.io outage

#55
I wake up to discover that my site https://remotehunt.com is super slow. First thing I do is visit HN to see what's up and I instantly see that Sentry is down.

I'm using Sentry to monitor logs and it now makes sense. Ok, so I remove Sentry from Laravel's error handler but nothing changes. And it's weird because sometimes it works, sometimes not. I tweak some things on Cloudflare (turning on Under Attack mode etc).

I still have this feeling that maybe it's related to Sentry: I'm using it and HN says it's down. So I go and remove the composer package as well, just in case. And it worked.

It took like an hour... I didn't use Sentry anyways :)

What did I learn: do not depend on external services if you really don't need it.

Re: Sentry.io outage

#56
I’ve used them for a number of years, never so much as a blip. Looks like they’re doing a good job managing this outage. They’re also super transparent with their bugs in their change logs. I recommend and love Sentry.

Re: Sentry.io outage

#57

I’ve used them for a number of years, never so much as a blip. Looks like they’re doing a good job managing this outage. They’re also super transparent with their bugs in their change logs. I recommend and love Sentry.

And they let you self host for free, although it is quite the process.

Re: Sentry.io outage

#58

I wake up to discover that my site https://remotehunt.com is super slow. First thing I do is visit HN to see what's up and I instantly see that Sentry is down. I'm using Sentry to monitor logs and it now makes sense. Ok, so I remove Sentry from Laravel's error handler but nothing changes. And it's weird because sometimes it works, sometimes not. I tweak some things on Cloudflare (turning on Under Attack mode etc). I…

Sentry's outage documentation implies that some underlying platform is the cause. Perhaps there's a major data center having issues?

Re: Sentry.io outage

#59

I wake up to discover that my site https://remotehunt.com is super slow. First thing I do is visit HN to see what's up and I instantly see that Sentry is down. I'm using Sentry to monitor logs and it now makes sense. Ok, so I remove Sentry from Laravel's error handler but nothing changes. And it's weird because sometimes it works, sometimes not. I tweak some things on Cloudflare (turning on Under Attack mode etc). I…

Sentry's outage documentation implies that some underlying platform is the cause. Perhaps there's a major data center having issues?

Maybe yes. It was very weird: even without sending any error messages to Sentry, it was still slow just by having the composer package installed. Some Sentry DNS thing? (https://docs.sentry.io/platforms/php/guides/laravel/#configu...)

Re: Sentry.io outage

#60

Obviously it's never good when a service has an outage, but it looks like Sentry's status page reports the actual status which makes a nice change.

If even their own operational telemetry were misrepresented, one might have started wondering about the accuracy of what they're reporting to their customers.

So, I agree, but it's not like they really had a choice :)

Post reply on HN