Live data from Hacker News

Twilio is erroneously over-billing and suspending accounts

status.twilio.com

51–60 of 123 posts

Re: Twilio is erroneously over-billing and suspending accounts

#51
post #39

For people commenting about overdraft charges, Dreamhost went through this a while back. I was in college at the time, and didn't realize they'd drafted a second charge to my account. A series of small transactions later, and I was $300 in the red with very little income to fix it. They offered to reimburse overdraft fees for those affected, but I never was reimbursed. This is also great as a "how not to tell custome…

What's wrong with the Dreamhost post? He specifically says: >"The end to this story is that of course, I’m very very sorry, we’re very very sorry, and I’m sure you’re very very sorry this happened. I really am. I understand the sort of problems that an unexpected large charge to your credit card (or worse yet, your debit card) can cause. If the tone of this blog post seemed a little light, I apologize I don’t mean to…

The bank was SunTrust. They were very scummy.

As for what's wrong with the post, a ton. This was e-mailed to every single customer. For many people, myself included, this was their first notice that something was wrong with their finances. And it was the author's fault. The line you quoted was at the end of a very long post. The apology should have been right in the first paragraph. Instead in the first two lines he's complaining about how bad this is for him.

Then he makes a bunch of jokes. Then he explains why it was just tiny mistake. Don't feel bad if I missed the problem? I'm not the one being trusted with millions of dollars of your customer's money... [Edit: to me that bit reads as "See? You missed it too! Now why should I feel bad?"]

After all this, finally, he apologizes.

For me, this guy's mistake meant that I now had to worry about how I was going to pay bills/rent, and how I was going to eat. My situation was probably quite unique, but that didn't make it any less impactful for me.

To me the whole thing reads as "I just accidentally removed a few million dollars from my customers banks/credit lines - so now I'm gonna have a laugh with them!"

I can say with great confidence that I'm not immune to the mistakes that caused this to happen. But what I can say is that were I in his shoes, I'd lead with an apology, be very succinct about what happened, and finally list details as to how I'm going to make it better.

Edit: To his credit, if you read the next post he did a much, much better job there.

Edit 2: And also for what it's worth - I still use Dreamhost. Aside from this SNAFU they've ran an excellent service, and (again, except in this case) their customer service has been excellent in responding to any issues I've had. However now I always pay with credit rather than debit, and I don't store my card info.

Re: Twilio is erroneously over-billing and suspending accounts

#53
post #2

Yeap had this ourselves. Auto recharge actually took $500 rather than $20!

Absolutely unacceptable.

As I indicated above Daniel, if you haven't shot a note to help@twilio.com indicating you're affected by the incident, please do so we can communicate directly when this $480 mistake is corrected.

Re: Twilio is erroneously over-billing and suspending accounts

#54
post #11

Earlier quoted context omitted.

Doesn't matter how good you think are. If you write code professionally for long enough, you eventually experience an event like this first hand. It's an absolutely horrible experience for a developer.

Being a professional developer is hard. That's why it's generally a high paying position, because you solve hard problems and are accountable for the results. So stop making excuses. An organization of this scale should be taking serious precautions (against code errors, bad hardware, malicious attacks, network problems, compiler bugs, cosmic ray induced bit errors, etc.) when building a credit card processor. If som…

I would think NASA has even more controls in place and still shit happens. All problems are avoidable in hindsight but it is inevitable that you will have problem at some point.

Re: Twilio is erroneously over-billing and suspending accounts

#55
post #30

This sort of thing has happened before to another company, but that was a case of human error compounded by the fact that there were no software safeguards in place. http://dreamhost.com/dreamscape/2008/01/15/um-whoops/ At the very least, it would be wise for any company to implement billing features that will under bill rather than overbill. Under billing at least gives you the option of notifying users later (or yo…

Unfortunately when automation goes wrong, it usually tends to do too much - double billing, aggressively canceling things, etc.

Having built several automation systems that bill millions of dollars, I can say that it's quite challenging to build things that have no visible output, run by themselves and often sit there for years without being touched, while everything changes around them!

It's not sexy work, but when it goes wrong the shit really can hit the fan!

Re: Twilio is erroneously over-billing and suspending accounts

#56

I think the fact of the matter is as a developer I want to get angry at them and then remember this is my own worst nightmare - mistakes happen, the true test will be in how Twilio reimburse people.

Yeah, I see a lot of (completely fair) angry comments in this thread. But honestly the response has been pretty damn good for what happened, and we will see how it plays out. Any one of us could have made this mistake and Twilio is a great service, otherwise you wouldnt have seen many people here complaining at all!

Re: Twilio is erroneously over-billing and suspending accounts

#57
I see a lot negative comments here. Frankly, I understand people's frustration, but I think Twilio are dealing with this in the right way.

I was one of the customers affected. Luckily for me, my credit card automatically blocked the transaction after the 10th time I was auto-recharged, meaning I had a cap on how much I paid. I can totally understand the frustration of people discovering that they suddenly have 100's or even 1000's of dollars worth of charges to their credit card.

On the other hand, by the time I noticed that something strange was going on, Twilio's status page already said they were looking into it. I contacted their support right away, and they answered within 20 minutes, during what must be a customer support crisis.

And now RobSpectre is on HN giving very helpful answers.

So, no, it's not their finest work (as RobSpectre has said), but at least they're dealing with it quickly, and hopefully professionally as well. Let's let this play out before we jump up and down on a great company that we all love.

Re: Twilio is erroneously over-billing and suspending accounts

#58
post #56

I think the fact of the matter is as a developer I want to get angry at them and then remember this is my own worst nightmare - mistakes happen, the true test will be in how Twilio reimburse people.

Yeah, I see a lot of (completely fair) angry comments in this thread. But honestly the response has been pretty damn good for what happened, and we will see how it plays out. Any one of us could have made this mistake and Twilio is a great service, otherwise you wouldnt have seen many people here complaining at all!

Response is not good until everyone affected has their money returned to them and we explain what happened.

Very much appreciate your support through a difficult night and morning. More to come.

Re: Twilio is erroneously over-billing and suspending accounts

#59
post #9

Wow. Seriously? Learn to code.

How is that constructive at all? Shit happens. My code has bugs, your code has bugs. Just about everyone has something break at some point. At least Twilio has some reasonable disclosure of these issues on their dashboard to inform people of what is going on. I'm sure they will rectify the issue as soon as they have everything back up and running.

Your code has bugs, my code has bugs, but when our organization fails to implement a system to effectively catch bugs - particularly catastrophic ones - it's a bit dismissive to throw a "shit happens" at it.

Re: Twilio is erroneously over-billing and suspending accounts

#60
Apologies for nit-picking at punctuation, but occasionally it matters :-)

"Twilio is over-billing ..." = Twilio is charging more than they should.

"Twillio is over billing ..." = Twilio has given up on billing. It's so over.

Just my personal bugbear - but sometimes it's worth getting this sort of thing correct. Apologies if this is just a US/UK difference, also (I'm in the UK).

Post reply on HN