Live data from Hacker News

Mandrill has been down for over 30 hours with no explanation

twitter.com

91–100 of 115 posts

Re: Mandrill has been down for over 30 hours with no explanation

#91
We moved off Mandrill back in 2016 when they changed their TOS effective immediately for one of their major use cases (sending newsletters).

I'm kinda glad now they forced the issue.

HN coverage of that event: https://news.ycombinator.com/item?id=11203056

Re: Mandrill has been down for over 30 hours with no explanation

#92
Wow, that's pretty bad. I'm sure a lot of companies were affected. No ETA on a fix is really scary.

We currently use Law Ruler to send out our emails and have had no issues in the last 2 years.

If anyone needs a new solution fast I would reach out to their support ( form on website ).

Re: Mandrill has been down for over 30 hours with no explanation

#93
It's a spamming service, right? (Not, as one might think from the name, a gay dating service.) Is it a big deal if it's down to anyone but the spammers?

There's an annoying tendency to combine spam services with actual transaction reports. (A transaction report is "Your order has shipped and here is the tracking number", not "We have a new product") Because everybody blocks the spammers. Constant Contact went down that road, and now you can't reliably use Constant Contact for transactions.

Re: Mandrill has been down for over 30 hours with no explanation

#95
post #27

Unfortunately this is standard MailChimp way of doing things ever since they screwed over paying customers and merged with Mandrill [1]. They are the opposite of a transparent organization - they are the GoDaddy of this business. (I was super pissed once they removed their status page..WTF). We hit these errors a few months ago with our clients and I put up a roadmap to move all of them off the terrible joke that Man…

Move to mailgun https://www.mailgun.com/ I was already angry enough when mailchimp merged mandrill with its main service

Loved mailgun when I was using it a few years ago for some small campaigns. Unfortunately the free tier service had a big problem with server reputation. A noticable amount of emails were spam-flagged or rejected by recipient systems.

Not surprising that free services would be abused, but was a little disappointing. I would assume the paid customers get servers with better reputation/history.

Re: Mandrill has been down for over 30 hours with no explanation

#96
post #44
post #27

Unfortunately this is standard MailChimp way of doing things ever since they screwed over paying customers and merged with Mandrill [1]. They are the opposite of a transparent organization - they are the GoDaddy of this business. (I was super pissed once they removed their status page..WTF). We hit these errors a few months ago with our clients and I put up a roadmap to move all of them off the terrible joke that Man…

I moved to sendgrid when they announced mandrill's merger and have never been happier.

SendGrid used to have a free plan but things have changed since being acquired. I moved to SendGrid anyways though and use their free 25,000 emails / month on Azure even though our app is hosted elsewhere:

https://azuremarketplace.microsoft.com/en-us/marketplace/app...

Re: Mandrill has been down for over 30 hours with no explanation

#97
post #68

Got this email just now. - - - - Hello, We’re contacting you about an ongoing outage with the Mandrill app. This email provides background on what happened and how users are affected, what we’re doing to address the issue, and what’s next for our customers. What happened Mandrill uses a sharded Postgres setup as one of our main datastores. On Sunday, February 3, at 10:30pm EST, 1 of our 5 physical Postgres instances…

If you care about scalability and availability simultaneously, I'm not sure in these modern times why you would use a relational database. When they fail, they fail catastrophically and are difficult to recover, as this failure event (and the never-ending stream of failure events posted to HN) demonstrates. Don't get me wrong--I love relational databases and they are amazing pieces of technology. But they are incredi…

https://aws.amazon.com/message/5467D2/

Re: Mandrill has been down for over 30 hours with no explanation

#98
A few of my teammates at SendGrid have been following the situation, and we definitely feel for the engineers who are scrambling to fix the problem. It's never fun to get paged, especially when the trust of your customers on the line.

Some folks on the email thread were personally involved in handling major outages in the early days. We've had to learn a lot of hard lessons since then. Even when everything seems like it's going fine ("wow, we're growing so fast, good problems!"), scaling issues could be right around the corner.

Anyone with a large enough installation of Postgres could've had the wraparound issue we're seeing right now. That's why it's important to monitor for what could go wrong, detect these issues early, and provide customers with rapid communication so they can plan around it.

Sending our best wishes to the MailChimp engineering teams working on the problem right now. Good luck, you've got this!

Re: Mandrill has been down for over 30 hours with no explanation

#99
post #2

Disclaimer: it's my tweet that is linked in this story, and I submitted it. There's no news story as far as I know. Better links are appreciated. Mandrill stopped delivering email at 04:51 UTC on the 4th of February. Nobody knows whether the received emails are lost, what the cause is, how soon any change will occur. It took Mandrill over 9 hours to acknowledge the issue. They are posting the same message over and ov…

> which prevents email delivery from compliant Danish email servers.

Any email server which validates that the remote host name matches the cert will be be very sad. Doesn't matter if it's compliant (to what btw?), in practice email server owners are sloppy.

Re: Mandrill has been down for over 30 hours with no explanation

#100
post #95

Earlier quoted context omitted.

Move to mailgun https://www.mailgun.com/ I was already angry enough when mailchimp merged mandrill with its main service

Loved mailgun when I was using it a few years ago for some small campaigns. Unfortunately the free tier service had a big problem with server reputation. A noticable amount of emails were spam-flagged or rejected by recipient systems. Not surprising that free services would be abused, but was a little disappointing. I would assume the paid customers get servers with better reputation/history.

What do you recommend? I'm on their lowest cost tier and am seeing some of the issues you describe.
Post reply on HN