Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: Heroku deleted my database with no warning

news.ycombinator.com

Post reply on HN
Print view View original

191–200 of 216 posts

Re: Tell HN: Heroku deleted my database with no warning

#191

Post by jfoster »

mst wrote: post #159
celestialcheese wrote: post #53

Yup - this happened to me too. It was wildly frustrating, but they did send few emails looking back - it was just a really small project and it wasn't on my radar. What pissed me off most is that I WAS paying for the account. I was paying $17/mo for redis and dynos, so there was an active card on file. Why not just start charging for the postgres db, and only delete if there's no active billing? Heroku was already a…

> Why not just start charging for the postgres db, and only delete if there's no active billing? Probably because the original terms you agreed to were written not anticipating Salesforce perpetrating this and so while I would not at all be surprised if the vast majority of customers in your position would've been entirely happy with it they probably didn't have a legal path to do so.

> they probably didn't have a legal path to do so.

That makes no sense. Most large companies change both agreements and prices via notification emails all the time. Probably even SalesForce does so.

Re: Tell HN: Heroku deleted my database with no warning

#192

Post by jfoster »

hyperman1 wrote: post #190

I am wondering more and more if any trust in a single cloud provider isn't simply an unacceptable risk. The power balance means that a minor error on their side is generally business-ending for you. This is different from almost any other supplier, which can generally be replaced, even if the business is on hold for a few days. So any data recovery plan that should be able answer: what if the relation with our cloud…

Even the seemingly independent ones can get acquired by Google and eventually killed.

Anyone remember Parse? It was quite successful until Facebook got hold of it and seemingly out of nowhere decided to destroy it.

Re: Tell HN: Heroku deleted my database with no warning

#193

Post by glenngillen »

brogrammernot wrote: post #107
glenngillen wrote: post #99

Earlier quoted context omitted.

> Salesforce on bought them as a cash generator As an early Heroku employee, this gave me quite the chuckle.

Same - wasn’t an employee but used it often in ~2013-2015 timeframe. I’m pretty sure the valuation was many many many multiples of any sort of revenue based metric, wasn’t it?

Yep.

Also re the timeline, the acquisition was announced in 2010. So your own usage was already a few years post.

Re: Tell HN: Heroku deleted my database with no warning

#194

Post by jfoster »

dboreham wrote: post #81

They were charging money for thing A, that connected to (free) thing B, and rather than just begin charging the same customer, using same credit card, for thing B, they deleted it. I would love to see the recording of that zoom meeting...

Is there any way for it to make business sense, or was it pure incompetence?

I wonder how many databases they deleted that they could have instead started charging for. Seems to be incredibly destructive for all involved. (SF shareholders, Heroku customers, etc.)

Re: Tell HN: Heroku deleted my database with no warning

#195

Post by johannes1234321 »

jbverschoor wrote: post #178
johannes1234321 wrote: post #109

Earlier quoted context omitted.

It's not that easy. Those databases might contain personal information which is protected under different privacy laws like GDPR, HIPAA and others and Heroku/Salesforce can not simply store that for longer than agreed upon and Heroku cancelling the account enables the retention period as the customer agreed upon as part of the T&C.

I wonder if you guys delete email addresses from people who ask to unsubscribe , or when I tell t you to delete my information, according to gdpr Probably not, because it’s ” difficult “

Many avoid it, but once they have to deal with a (ex-)customer's lawyer they learn what is even more difficult.

Re: Tell HN: Heroku deleted my database with no warning

#196

Post by Cheezewheel »

throwawaykai wrote: post #21
ericpauley wrote: post #4

This whole debacle has been such short-term thinking from Salesforce. That after carrying these free projects for years they couldn't stomach more than 30d of data retention is just the icing on the cake. In my view this has caused yet further reputational harm for Heroku, and is going to have a long-term effect on the bottom line from paid projects. The value prop of Heroku has always been being able to sleep at nig…

Herokai here. Unfortunately we had no choice on the data retention front — once we’ve disconnected your database, we aren’t ALLOWED to hold your data for more than 30 days. That’s part of the data scrubbing protocol that we agree to when you sign up. We fought hard for 90+ days internally, but in the end couldn’t get over the issue that we’d be in violation of our contracts with customers.

No choice? That's just the way it is, it can't be helped? Did God himself come down and decree that Herokai *MUST* only hold data for 30 days? Did the FBI come in an threadten to charge your executives with sedition?

Yea, no. You decided to make the decision for contracts to be that way. The fact that you "fought hard" but that decided on the 30 day retention anyways means that clearly the opinions of engineers don't matter and that the company is completely captured by the lawyers and out of touch executives. It hardly inspires confidence.

It also doesn't at all address the fact that you failed to contact an apparently paying customer that their data was about to be nuked, contract or no.

Re: Tell HN: Heroku deleted my database with no warning

#198

Post by account42 »

hot_gril wrote: post #127
fishnchips wrote: post #119

Earlier quoted context omitted.

Very much that. As a former SRE responsible for tape storage, I saw things like regular backups of an error phrase "You have no access to this database". Guess what happened when the team accidentally dropped the database? Unless you're doing regular restores, you don't have a backup. You have hope. So yes, doing backups in a way that gives you some form of guarantee isn't exactly cheap.

Or we do have everything in the backup, but the restore process isn't worked out. Someone is losing a weekend writing hacky scripts, and every SLO is being violated, if we ever have to use it for real.

That case is still a lot better than no backups and having to tell your customers that the data is gone.

Re: Tell HN: Heroku deleted my database with no warning

#199

Post by gwbas1c »

m00x wrote: post #13
gwbas1c wrote: post #2

What frustrated me about Heroku nuking free projects was that the paid version was, IMO, too expensive. (Granted, I understand that the free tier was probably a money sink for them.) $5-10 a month is pocket change in some contexts, but for a hobby or a one-off project, it's a huge chunk of change. I really wish they offered a lower-cost tier.

That's 2 cups of coffee, or 1 latte. It's objectively not a huge chunk of change unless you live in a very LCOL area. You can also go to any competitors, which have their smallest tier around the same price. Linode has a $5/m tier. You could also buy a Raspberry Pi for $50 and run it in your closet.

> or a hobby or a one-off project

I don't want to pay $5 / month indefinitely to host a weekend project.

I don't want to deal with the complexities of hosting a weekend project myself. I've done plenty of that.

I just don't want to end up spending $100-200 / month to host a lifetime of weekend projects and experiments.

Re: Tell HN: Heroku deleted my database with no warning

#200

Post by AlisdairO »

hot_gril wrote: post #121
Andys wrote: post #118

Earlier quoted context omitted.

Yes, AWS is similar with their DB offerings. You can discourage it from doing any updates/reboots (which causes a failover), but ultimately if they want to failover, they can at any time.

Ouch. If it knows it's about to fail over from an update, it really should get the follower totally in sync with the leader first.

HA on rds uses synchronous replication - you won’t lose data on automated failover under any normal circumstances.
Post reply on HN