Live data from Hacker News

"Hetzner decided to cancel our account and terminate all servers"

mastodon.social

161–170 of 401 posts

Re: "Hetzner decided to cancel our account and terminate all servers"

#161
post #92

Earlier quoted context omitted.

They are great for throwaway hobbyist side projects where you don't want to worry about AWS billing horror stories or more expensive offerings like Digital Ocean or Linode. I would not recommend them for a serious, money-on-the-table business.

I only use them for money-making projetcs. Based on my own experience and what I read online, you need to be careful with: * crypto mining (I used it when it wasn't causing much trouble but I noticed my nodes were constantly attacked at a ratio I newer saw for other servers); IIRC Hetzner's current ToS forbid crypto mining * things in legally grey area which might be legal in some places but not so in others, especia…

Their ToS forbids not just the crypto mining (that was extremely reasonable to ban ten years ago, but it's moot today) but also some arbitrary financial technologies they don't like.

So beware of their ToS.

Re: "Hetzner decided to cancel our account and terminate all servers"

#162
post #70

Hang on, Hetzner literally deleted all their data without warning? That’s actually insane and business killing. Both for Hetzner’s reputation and potentially for their customer.

> Hang on, Hetzner literally deleted all their data without warning? That’s what the post said. But of course we have no idea if it’s true or not. No evidence was provided, and we are only hearing one side of the story.

No evidence was provided also because they did not send any email or offer any kind of statement,, pdf, anything, to explain what they did. It was a purely silent delete.

And in that hackernews thread we have dozens of people relating similar stories.

Re: "Hetzner decided to cancel our account and terminate all servers"

#163
post #14

Most of the times you hear people complaining about Hetzner shutting down someone's servers, it's because they were hosting content going against their ToC or similar. But this seems to be about Kiwix (which in short is "offline Wikipedia" in various ways) and doesn't seem to be about questionable content in any way. Eventually I guess we'll get Hetzner's perspective on this, as they tend to start writing publicly ab…

> Besides Wikipedia, content from the Wikimedia Foundation such as Wikisource, Wikiquote, Wikivoyage, Wikibooks, and Wikiversity are also available for offline viewing in various different languages. [0]

> Users first download Kiwix (or a browser extension), then download content for offline viewing with Kiwix. [1]

> Our main storage backend became entirely unreachable. For the average user that meant not being able to access the library and download files, and for us that meant not being able to connect to it and see what was wrong. [2]

Maybe some odd photos landed on WikiMedia which then got automatically synced to Hetzner's servers and then triggered some alarms.

I can't judge about Hetzner deleting the data, but them not attempting to really get in touch with the Kiwix team -- after all they should know that they are trying to do some good in this world -- is a really horrible move. In the same category as Google blocking access to user's accounts without any word, or German companies suing security researchers for notifying them about a security flaw in their systems.

Shame on Hetzner.

[0] https://en.wikipedia.org/wiki/Kiwix#Available_content

[1] https://en.wikipedia.org/wiki/Kiwix#Description

[2] https://mastodon.social/@kiwix/113622081750449356

Re: "Hetzner decided to cancel our account and terminate all servers"

#164

Whenever I ask a CTO if they have a backup (or plan-B) they say we're on AWS, we backup there and they will never go down as a company. And then I ask them what they do when their account gets shut (e.g. because they are selling something bad on Amazon and have the same phone number as the company account?) Or the instance some years ago where GCP closed because someone had wrongly classified image on their drive? Yo…

There should just be a legal duty placed on cloud providers to not do this. Nobody would expect you to hold a second redundant commercial lease for your offices or retail location.

I think this is a tough problem, partly due to the post-paid nature of most cloud services, partly due to the impact to other customers.

If you had a bunch of retailers in a shared space (like a market), and one of them was setting off fireworks, using all the power/water in the space, and scaring away customers, I'd expect them to get kicked out pretty quickly.

Now it may be that this is a false positive, I'm sure they happen, but in the case where it's a legitimate bad actor that is actively harming both the company and other customers on those servers, what's the course of action the company should take?

Re: "Hetzner decided to cancel our account and terminate all servers"

#165
post #22

Whenever I ask a CTO if they have a backup (or plan-B) they say we're on AWS, we backup there and they will never go down as a company. And then I ask them what they do when their account gets shut (e.g. because they are selling something bad on Amazon and have the same phone number as the company account?) Or the instance some years ago where GCP closed because someone had wrongly classified image on their drive? Yo…

Another great question is "When did you last try to restore from a backup?" which usually is answered with "It's the built-in tooling, why would we assume it's broken?" or similar. Then fast-forward some months/years, and they try to restore from backups only to realize the backups never actually backed up what they cared about.

This famously happened at GitLab: https://about.gitlab.com/blog/2017/02/01/gitlab-dot-com-data...

> Regular backups seem to also only be taken once per 24 hours, though team-member-1 has not yet been able to figure out where they are stored. According to team-member-2 these don’t appear to be working, producing files only a few bytes in size.

Re: "Hetzner decided to cancel our account and terminate all servers"

#166

Earlier quoted context omitted.

There is a huge amount of protection for renters, a lot of things are simply illegal to put into the rental agreement and are automatically void. I really have no idea what you're talking about here.

>and are automatically void And yet they're still put ion the rental agreement because the landlords know they can get away with it s it's a seller's market. >I really have no idea what you're talking about here. Google or look on reddit posts of foreigners getting screwed in Germany.

Foreigners are typically getting screwed in Germany precisely because they don't know their rights or where to go to ask for (free) legal advice.

If your landlord puts something in the contract that is against the law you can sign it and simply ignore it.

Re: "Hetzner decided to cancel our account and terminate all servers"

#167
post #14

Most of the times you hear people complaining about Hetzner shutting down someone's servers, it's because they were hosting content going against their ToC or similar. But this seems to be about Kiwix (which in short is "offline Wikipedia" in various ways) and doesn't seem to be about questionable content in any way. Eventually I guess we'll get Hetzner's perspective on this, as they tend to start writing publicly ab…

>But reading about people having their servers deleted in the middle of the night on a Sunday (Berlin time) and all data wiped immediately, with no recurse, does sound a bit aggressive.

There are several comments under this thread from people reporting essentially that happening to them.

Re: "Hetzner decided to cancel our account and terminate all servers"

#168

Earlier quoted context omitted.

One thing I've never figured out is, what is the difference between backups and replication? And, does restoring from backups always mean losing more _recent_ data than replication?

Replication isn't a backup, because if you accidentally delete a file, and that deletion is immediately replicated, then you can't get that file back. Backups are a specific point in time.

Backup is a useless word, it's too overloaded

Are snapshots backups? snapshots on raid? snapshots on replicated disks?

Re: "Hetzner decided to cancel our account and terminate all servers"

#169
Backups are irrelevent here (yes, backups are important). if hetzner really deleted production data without warning or providing a grace period for their customer to migrate their data, then they are simply not a stable foundation to build on.

I have never been a customer of google cloud for this reason and i sure as hell wont deploy new servers on hetzner until they provide a clear statement on what went wrong and what they will do to make sure they never screw up like this again.

Hetzner, the ball is in your court.

Re: "Hetzner decided to cancel our account and terminate all servers"

#170
post #147
post #21

hetzner is cheap, but cheap often has hidden costs / risks. aws, azure, and gcp aren’t cheap , but they offer better stability—both technically and operationally.

And service. AWS has lots of problems. But they have a team of real humans that respond to tickets around the clock and actually understand stuff. To many businesses, that's worth the extra cost clone.

Yep. Obviously YMMV, but I found AWS TAM to be much better than Google Cloud's (but that's not surprising).
Post reply on HN