Live data from Hacker News

Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

medium.com

11–20 of 60 posts

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#11

"I am single, 41 and pregnant"... I don't know when pregnancy became a part of a person's identity. I guess this adds to the coolness factor these days because you have to try harder? Sigh.

The pregnancy is just as relevant as being single and 41. Why are you complaining about it and not the others?

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#12

This could be solved an even more fundamental way: Don't run your own mailer as a startup. There are lots of companies that will be responsible for email deliverability on your behalf, via an API. If it took them 3 months to notice no mail was being sent at all, imagine how long it's going to take them to figure out that their IP is blacklisted in Spamhaus or any number of other deliverability issues?

Not everyone wants to outsource transactional mail handling to a third party.

Just because you don't want to doesn't mean you shouldn't.

The major advantage to outsourcing it is that the people you're handing the job over to actually know what they're doing and are experts at it. Plus they can spend 24 hours a day checking this stuff instead of you.

The arguments for being in-house on absolutely anything but your core competency when you're an early-stage startup are really hard to justify.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#13

"I am single, 41 and pregnant"... I don't know when pregnancy became a part of a person's identity. I guess this adds to the coolness factor these days because you have to try harder? Sigh.

Pregnancy at 41 is generally high-risk. So you have lots of doctor's appointments, often lots of discomfort and you probably end up with mandated (and necessary) bed rest.

Being single while dealing with that is an enormous burden mentally and physically.

I didn't notice it referenced in the story, but it is definitely a factor that would influence a person's behavior that would be relevant, especially in a startup context when everyone is wearing 10 hats.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#14
post #4

Earlier quoted context omitted.

Not everyone wants to outsource transactional mail handling to a third party.

If someone wants to do that then they've never tried to handle even a moderate volume of transactional mail themselves. Or they're starting a transactional mail service provider. Either way, my hat's off to them. It's a giant pain in the ass that can be resolved for less than $20 per month.

I completely agree. I think unless your startup or core competency is something like that, why shard your core product's development/IT/whatever work to support an in-house product that's bound to have bugs(tm) right out of the gate? There very much should be a cost/risk analysis but there's an awful lot of NiH that honestly should be avoided if you can genuinely help it. You're still more than welcome to roll your own everything but you're just delaying your core competency to maintain something you could offload to someone that specializes in X, Y, or Z.

I would also suggest if you do go that route, have accurate metrics on everything. How much time are you spending maintaining what essentially is a separate product? How much is it taking from your MVP or is it part of it? If you can translate that to cost then you can judge how much offloading that service would be to someone that specializes in it.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#16
As someone who has created quite a few of these mailers, the queue getting stuck on a single piece of mail and hanging indefinitely is incredibly common. As time has gone on my solutions have become simpler and more pragmatic, since additional complexity breads additional problems.

For example, if I was going to design an emailer today:

-Grab the email from a database save it to a file (likely one or several XML files) and place it in an "Outgoing" directory (ye olde file system).

- Then have a process which grabs an atomic lock (only one running at a time!), gets the directory listings, and launches the actual "sender" for every file individually (concurrently).

- When the launcher launches the sender it records the PIDs of the process against the actual emails/XML files internally.

- After a set wait period if any processes are still running, the launcher kills them, and moves the email/XML into a "Failed" directory which we monitor independantly.

- Every email which is sent gets moved to an "Archive" directory by the sender process, and we monitor that to see if no emails have been archived for a long time (e.g. 30 minutes).

You can accomplish the same thing using a database (Outgoing, Achive, and Failed tables), but frankly with so many awesome file system tools already around it doesn't make sense to reinvent that wheel. Plus people intuitively understand that if a file is sitting in the "Outgoing" or "Failed" directories then it hasn't been sent yet (just like your client would!).

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#17
I don't think terms like "Fuck-ups" or "screwed" should belong in corporate communications, start-up or not. It's cool they are talking about this openly, but unfortunately what I took from their write-up is that their communication style is less than professional.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#18

This could be solved an even more fundamental way: Don't run your own mailer as a startup. There are lots of companies that will be responsible for email deliverability on your behalf, via an API. If it took them 3 months to notice no mail was being sent at all, imagine how long it's going to take them to figure out that their IP is blacklisted in Spamhaus or any number of other deliverability issues?

If you want to send bulk email blasts from a large list then definitely use a 3rd party, but it seems their emails were triggered programmatically from many different parts of their application. The email providers I have used don't make this easy so it is usually better to run the server yourself. You don't need much knowledge or experience to run an email server (if you are competent at general IT tasks already) but you need one thing: to actually check once in a while to see if your sever is sending email.

Which is the real problem here. Team members knew mail wasn't being delivered in the forums and they chose to ignore it. They must have never done any follow-up (personal email, phone call, survey) on new customers even when they were doing their big marketing "ramp-up." They must not have even checked with a test walk-through of the new user process. Leadership was just too far removed from the customer experience, whether they used a 3rd party email service or not.

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#19

This could be solved an even more fundamental way: Don't run your own mailer as a startup. There are lots of companies that will be responsible for email deliverability on your behalf, via an API. If it took them 3 months to notice no mail was being sent at all, imagine how long it's going to take them to figure out that their IP is blacklisted in Spamhaus or any number of other deliverability issues?

If you want to send bulk email blasts from a large list then definitely use a 3rd party, but it seems their emails were triggered programmatically from many different parts of their application. The email providers I have used don't make this easy so it is usually better to run the server yourself. You don't need much knowledge or experience to run an email server (if you are competent at general IT tasks already) bu…

I'm not sure I understand. Why couldn't they use a 3rd party hosted smtp server to send mail programmatically?

Re: Startup Fuck-ups: How we lost 25% of our monthly revenue overnight

#20
post #17

I don't think terms like "Fuck-ups" or "screwed" should belong in corporate communications, start-up or not. It's cool they are talking about this openly, but unfortunately what I took from their write-up is that their communication style is less than professional.

I don't care what language they use, personally. However I am at work and having the F-word in 72pt font blaring from the top of my browser made scroll down very quickly. I doubt my bosses would be thrilled to see that.
Post reply on HN