Live data from Hacker News

Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

medium.com

41–50 of 60 posts

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#42

This could be solved an even more fundamental way: Don't run your own mailer as a startup. There are lots of companies that will be responsible for email deliverability on your behalf, via an API. If it took them 3 months to notice no mail was being sent at all, imagine how long it's going to take them to figure out that their IP is blacklisted in Spamhaus or any number of other deliverability issues?

I don't entirely agree with this.

I don't use my own mailer currently. I DO, however, use postfix to queue and relay email to rackspace, who actually sends my email.

I don't think the API method is appropriate because then you need to run some other queue system so your app has an instant response time for the user. Their action would create a queue entry (with whatever data) that will eventually be fired off as an API call to whoever you're using to send email via an API.

Or, you just set up an SMTP relay and use sendmail/postfix/whatever locally to handle that part of it.

I often see these startups using the API calls as part of the customer facing flow (website or otherwise) and the increased latency waiting on that API call to return really, really sucks.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#43
post #30

Earlier quoted context omitted.

> The pregnancy is just as relevant as being single and 41 ie. not at all

All are fairly relevant in the user's profile , which is where this appeared.

You are being quite naive if you don't understand why it's there.

When is the last time someone introduced themeselves to you as: "hi, i am xx and i am pregnant"? It makes no sense whatsoever.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#44
post #40

What we found was that a number of failed jobs were being kept by the system, which meant that these were taking up a ton of space that they shouldn’t have been. To fix the issue, we put together a script to delete the failed items, since any retries to send them didn’t appear to work. At this point my head was screaming "NOOOOooooo!" and made me feel bad for author for the whatever disaster would soon follow. Not on…

Exactly what I thought!

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#45

This could be solved an even more fundamental way: Don't run your own mailer as a startup. There are lots of companies that will be responsible for email deliverability on your behalf, via an API. If it took them 3 months to notice no mail was being sent at all, imagine how long it's going to take them to figure out that their IP is blacklisted in Spamhaus or any number of other deliverability issues?

Focus on your core competency, as someone who has run production mail servers in the past I know it's not something you really want to do (My advice: don't) there is always some fire to put out or blacklist you were put on that needs to be cleaned up.

Mailgun offers 10,000 emails per month free and is dead simple to use.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#46
post #34

"Small" is relative -- at the "small startup" I run, our email volume is low enough that I BCC myself on every email the site sends. Poor man's monitoring and it won't scale, but I usually notice within hours if something had been broken.

Where I work, BCC-ing on all mails is not scalable. Instead a random sample of the mails is sent to ourselves.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#47
post #42

This could be solved an even more fundamental way: Don't run your own mailer as a startup. There are lots of companies that will be responsible for email deliverability on your behalf, via an API. If it took them 3 months to notice no mail was being sent at all, imagine how long it's going to take them to figure out that their IP is blacklisted in Spamhaus or any number of other deliverability issues?

I don't entirely agree with this. I don't use my own mailer currently. I DO, however, use postfix to queue and relay email to rackspace, who actually sends my email. I don't think the API method is appropriate because then you need to run some other queue system so your app has an instant response time for the user. Their action would create a queue entry (with whatever data) that will eventually be fired off as an A…

Just wondering: would it work if your app provider supported email submission via Job queue (e.g. Amazon SQS/Iron MQ) ?

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#48

As someone who has created quite a few of these mailers, the queue getting stuck on a single piece of mail and hanging indefinitely is incredibly common. As time has gone on my solutions have become simpler and more pragmatic, since additional complexity breads additional problems. For example, if I was going to design an emailer today: -Grab the email from a database save it to a file (likely one or several XML file…

Sounds like you just recreated Microsoft Biztalk. But to sound less like a dick - communicating with a low probability of errors is hard !

As someone who's actually used Biztalk quite extensively: You wish Biztalk was that straightforward.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#49
post #47
post #42

Earlier quoted context omitted.

I don't entirely agree with this. I don't use my own mailer currently. I DO, however, use postfix to queue and relay email to rackspace, who actually sends my email. I don't think the API method is appropriate because then you need to run some other queue system so your app has an instant response time for the user. Their action would create a queue entry (with whatever data) that will eventually be fired off as an A…

Just wondering: would it work if your app provider supported email submission via Job queue (e.g. Amazon SQS/Iron MQ) ?

Yes, it would. In that case you are also punting on the job queue responsibilities and using a third party for both email and queueing jobs. That's not necessarily a good thing or a bad thing - it's just a thing.

You'll want to queue those API calls so your queue mechanism returns to the user very quickly, and then the emails can fire out .5-5 second later.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#50
post #12

Earlier quoted context omitted.

Not everyone wants to outsource transactional mail handling to a third party.

Just because you don't want to doesn't mean you shouldn't. The major advantage to outsourcing it is that the people you're handing the job over to actually know what they're doing and are experts at it. Plus they can spend 24 hours a day checking this stuff instead of you. The arguments for being in-house on absolutely anything but your core competency when you're an early-stage startup are really hard to justify.

Especially with the number of services that offer zero-cost plans at low usage levels! You can rarely even make a fiscal argument for keeping this kind of stuff in-house, much less a skills argument.
Post reply on HN