Live data from Hacker News

Tell HN: Heroku deleted my database with no warning

news.ycombinator.com

131–140 of 216 posts

Re: Tell HN: Heroku deleted my database with no warning

#131

This happened to me as well: projects with paid dynos but free databases, databases got nuked. How Heroku missed this is beyond me. They managed to screw over paying customers in their broad attempt to stop freeloaders. These are good accounts with credit cards on file. Why not just autocovert me to the lowest tier paid database? FWIW I was able to get them to restore my databases. But I also had free Heroku Redis on…

Redis is a memory store, you're the one who fucked up treating a cache as a persistence layer.

[deleted]

Re: Tell HN: Heroku deleted my database with no warning

#132

This whole debacle has been such short-term thinking from Salesforce. That after carrying these free projects for years they couldn't stomach more than 30d of data retention is just the icing on the cake. In my view this has caused yet further reputational harm for Heroku, and is going to have a long-term effect on the bottom line from paid projects. The value prop of Heroku has always been being able to sleep at nig…

[deleted]

Re: Tell HN: Heroku deleted my database with no warning

#133
Something similar happened to me on IBM Cloud, back when it was called Bluemix. This was early in my career for an early-stage startup (that eventually shuttered - not due to this issue, but rather due to a lack of investor interest and us running out of runway as a result). Easy enough setup: staging and production, each with two Docker containers, one for the (Elixir/Sugar) app and one with the (Postgres) database.

Well, one day the production Postgres container just... vanishes. All the storage is gone. After weeks back and forth with IBM's support, they confirm that the loss is permanent. No explanation, no refunds, just "lol fuck you". Naturally, we didn't have backups yet, so the few users we did have now had to start from scratch.

I, too, learned my lesson about not doing an offsite backup regularly - and the bigger lesson that Bluemix should've been a last resort option. Funny enough, we had migrated to that from Heroku; we ended up migrating again to AWS (specifically: Elastic Beanstalk).

Re: Tell HN: Heroku deleted my database with no warning

#134
Same here. I have 2 big gripes:

- I did not receive any email warnings that this would happen. I have a valid email registered with them because I do receive non-promotional emails. However nothing for this. After getting in contact with customer service their rebuttal was that I also would've known by checking the forums/blog. Who on earth is doing that? It's a terrible response.

- They detached the DBs at the start of December. I wasn't going to fix my broken personal projects over the holidays. I told myself I would take a look come January. Which by that point it was past the 30 day grace period.

In the end, I've had to completely stop paying for the dynos. So Heroku lost business overall from me.

Re: Tell HN: Heroku deleted my database with no warning

#135
post #126

Earlier quoted context omitted.

Check out render.com, it's a solid alternative.

We currently use render. Its excellent, but is missing a couple key features: like Heroku's continuous rollback protection. If they offered this they'd be head and shoulders better.

(Render CEO) We're actively working on Point-in-Time Recovery and hope to launch it in early access soon.

Re: Tell HN: Heroku deleted my database with no warning

#136

Earlier quoted context omitted.

Herokai here. Unfortunately we had no choice on the data retention front — once we’ve disconnected your database, we aren’t ALLOWED to hold your data for more than 30 days. That’s part of the data scrubbing protocol that we agree to when you sign up. We fought hard for 90+ days internally, but in the end couldn’t get over the issue that we’d be in violation of our contracts with customers.

Having worked at Heroku and had a large part in building Heroku Postgres I do not recall this explicit policy, and it seems very squirrelly to me. Maybe this came in as a policy in recent years and it is the case, but still seems like hiding behind a policy as opposed to doing right by customers. You could easily block all incoming connections to the database. For a free database of 10k rows there were no SLAs, and y…

[deleted]

Re: Tell HN: Heroku deleted my database with no warning

#137

Earlier quoted context omitted.

Are you allowed to inform paying customers that you are going to do this? This is my primary complaint here. I don't understand how this oversight happened. This is going to cause an enormous amount of time and energy to recover from this.

> Are you allowed to inform paying customers that you are going to do this? I can't be the only one who's basically completely blind to emails from major companies, including SaaS providers, because they're so fucking spammy that the SNR is like 1:99. Notifying me by email, for one of these places, is functionally the same as not notifying me at all. [EDIT] Sorry, didn't mean to imply the parent wasn't paying attenti…

> ...because they're so fucking spammy that the SNR is like 1:99.

This 1000x. I signed up for an SMS gateway service last year. Just for my own hobby use, nothing major. I gave them $10 to start service, and they charge like 2¢ or something per outgoing message.

They have like 180 different prices for 180 countries, territories, provinces, parishes, cantons, prefectures, etc. Every week one of those prices changes, and I get an email notifying me of that. I tried to turn those off in my preferences, but they refused. I can opt out of marketing, weekly digests, and "tips and tricks"; but I can't opt out of pricing notices.

So I added a rule on my end to hide those. I totally understand where they're coming from. They can't NOT give pricing change warnings. But at the same time, in the flood of constant notices, there may be something major I will miss.

It would be nice if they instead gave me the option to never spend more than $0.XX per message, and return an API error if an attempted send fails for price threshold violation. Then the spam wouldn't be needed.

Re: Tell HN: Heroku deleted my database with no warning

#139
post #112

Earlier quoted context omitted.

The customer account wasn't terminated, the free DB being used by paid Dynos was deleted without any input from or notification to the customer. I highly doubt normal customers want this clock short when the cancellation is not customer initiated.

Data retention policies are written by compliance people, not product people, so distinctions like overt, deliberate cancellation and "cancellation for nonpayment" or "abandonment" or "discontinuation" usually aren't captured in them. I'm not saying it's great that Heroku deleted these databases; I'm saying: the description given upthread, that the databases were deleted because of contractual requirements, is super…

If you start disconnecting the database, in the same way a user could (on most services?) to keep the database around and idle, there's no way that counts as any kind of deletion. So do that if you need to and you really want to give customers 90 days to react.

Re: Tell HN: Heroku deleted my database with no warning

#140
Bought some HDDs with some left over end of quarter money, found an old dell server, slapped em in installed it in a switch closet, and have the cloud perform backups to this local closet server. I even do this at home. I don't trust these guys with anything. Sorry for your loss
Post reply on HN