Live data from Hacker News

Twilio is erroneously over-billing and suspending accounts

status.twilio.com

91–100 of 123 posts

Re: Twilio is erroneously over-billing and suspending accounts

#91
post #82

Earlier quoted context omitted.

Not a single person was confused by this, including me, a British dude sitting in the middle of London.

Now you got me confused. You are saying that no one was confused, not even you. Or you say that it was confusing not only for a single person, but for you too?

I'm happy to take the karma hit I'll get for this, but IMO this is emblematic of a large problem with HN, i.e. asinine bike-shedding. I address an unhelpful nitpicking comment and get a nitpicking reply. It's turtles all the way down.

Re: Twilio is erroneously over-billing and suspending accounts

#92
I work in telecom, in a slightly different vein from Twilio's and I deeply empathize with the pain of billing.

Even if Twilio's somehow able to ignore the pain of telco engine rating and least-cost-routing, there are still so many places you can make an error and cause something like this.

Kudos to Rob and team for owning the issue and I hope the Twilio folks get this sorted out (I'm confident they will). It's worth mentioning that I've never seen another billing issue of this size from Twilio so one in 3-4 years isn't too awful. Compared to AT&T it's probably generous! (Facetious but with a grain of truth).

Good luck guys and keep rocking. Here's hoping everything works out.

Re: Twilio is erroneously over-billing and suspending accounts

#93
post #60

Apologies for nit-picking at punctuation, but occasionally it matters :-) "Twilio is over-billing ..." = Twilio is charging more than they should. "Twillio is over billing ..." = Twilio has given up on billing. It's so over. Just my personal bugbear - but sometimes it's worth getting this sort of thing correct. Apologies if this is just a US/UK difference, also (I'm in the UK).

agreed. I was excited to see how they were getting around that pesky issue of billing people.

same here. I was hoping the story introduced a new paradigm of charging that eliminated 'suspending' accounts all together, such as pre-paid.

Re: Twilio is erroneously over-billing and suspending accounts

#94
post #80
post #30

This sort of thing has happened before to another company, but that was a case of human error compounded by the fact that there were no software safeguards in place. http://dreamhost.com/dreamscape/2008/01/15/um-whoops/ At the very least, it would be wise for any company to implement billing features that will under bill rather than overbill. Under billing at least gives you the option of notifying users later (or yo…

Here's a choice piece of code we have in production along the lines of what you suggest: if prorated_amount > (cost_per_month * 2) raise "critical! we were about to prorate-charge someone #{prorated_amount}" end It's never been triggered, but this is probably the only place in our system where we're actually responsible for an exact amount that ends up on an invoice, so having a reasonable upper limit seems like a no…

shouldn't it be >=, to catch double-charging?

Re: Twilio is erroneously over-billing and suspending accounts

#95
post #14

Haha, the one time in my life having an outdated credit card on file paid off.

As uptown said, providing valid credit card information creates an authorisation for a merchant to charge to you, if your card expires after that authorisation has already been created you can still be charged and that charge will still be approved by your credit card provider. This is a common problem for people who take out short term loans and then think they can cancel their card to escape being emptied out when…

Also there are automated credit card update services, so if you get a card reissued then vendors can get their card on record updated.

Re: Twilio is erroneously over-billing and suspending accounts

#96
post #5

I'm glad they (and most of us) are not in the business of making insulin pumps or other software where such bugs could have potentially lethal consequences.

Unfortunately you use very different software development practices which are designed to move slower for things where lives are on the line. Things more akin to Spiral and Waterfall.

I'm not 100% sure that "unfortunately" is how I would describe an approach that is explicitly deliberative and slow-moving, when for instance devices are implanted into people's bodies.

Re: Twilio is erroneously over-billing and suspending accounts

#97
post #80
post #30

This sort of thing has happened before to another company, but that was a case of human error compounded by the fact that there were no software safeguards in place. http://dreamhost.com/dreamscape/2008/01/15/um-whoops/ At the very least, it would be wise for any company to implement billing features that will under bill rather than overbill. Under billing at least gives you the option of notifying users later (or yo…

Here's a choice piece of code we have in production along the lines of what you suggest: if prorated_amount > (cost_per_month * 2) raise "critical! we were about to prorate-charge someone #{prorated_amount}" end It's never been triggered, but this is probably the only place in our system where we're actually responsible for an exact amount that ends up on an invoice, so having a reasonable upper limit seems like a no…

What if this code is called multiple times in quick succession? Based on the fact that I got a dozen separate emails over the span of 10 minutes, I'm guessing Twilio's overcharging happened in a series of transactions, not all at once.

This code -- while good for catching bad single calculations -- wouldn't catch that.

Re: Twilio is erroneously over-billing and suspending accounts

#98

Rob from Twilio here. We are still actively working on this incident and aware that erroneous auto-recharge billing and account suspensions have occurred for a number of customers. If this has happened to your account, please send an email to help@twilio.com where we have our support team working on making this right for everyone affected. We are spinning up additional resources to make sure every customer issue gets…

Peter Gibbons: You know, corporate accounting is sure as hell gonna notice $305,326.13, Michael!

Re: Twilio is erroneously over-billing and suspending accounts

#99
post #43

Earlier quoted context omitted.

I was about to say this too. There was a lot of vitriol on the comments as well, but all of them follow the frustration of being overdrawn. But I agree, if the bank or card company doesn't take this as a genuine error and erase the record, then it's a pretty lousy institution. Maybe he could have worded it differently and shown less irreverence, but that may be a byproduct of the stress as well. It's a pretty shockin…

See my other comment for thoughts on the e-mail/post (sibling to yours). On the banks... "Overdraft protection" is probably the single most disingenuous thing the banks have done to low-income individuals. It's such a "squeeze 'em when they're down" tactic. I had all kinds of financial problems in college, mostly my own doing, but they were sooo complicated by this... service. Suddenly because of one mistake, 10 smal…

My local college credit union has an optional "Overdraft Protection Loan" service. You apply for an $X loan in the usual way (so bad credit, etc. can still hurt you here), but if/when you get it, they don't give you $X cash. Instead, they attach it to your checking account. If you go over, the bank automatically moves money from the loan to your checking account in $100 increments to cover the overdraft. There's no per-use fees, and as long as you pay it back within a month, there's no interest charged. (Or very little, I'm not clear--sometimes there's an extra 40 cents or so on there.)

It's really, really handy and they don't advertise it at all. I found out about it when I accidentally ran up a dozen overdraft fees and talked to an account manager about how to keep it from happening again. I'd recommend that everyone see if their bank offers something similar.

Re: Twilio is erroneously over-billing and suspending accounts

#100
post #5

I'm glad they (and most of us) are not in the business of making insulin pumps or other software where such bugs could have potentially lethal consequences.

Yes the machine producing insulin pumps would be in a state of continuous integration/deployment. Then one day at 3am a developer on a binge of Monster Energy and pizza would push an unchecked bug to git and have it deployed.
Post reply on HN