Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Hetzner decided to cancel our account and terminate all servers"

mastodon.social

Post reply on HN
Print view View original

171–180 of 401 posts

Re: "Hetzner decided to cancel our account and terminate all servers"

#171

Post by dawnerd »

diggan wrote: post #22
KingOfCoders wrote: post #19

Whenever I ask a CTO if they have a backup (or plan-B) they say we're on AWS, we backup there and they will never go down as a company. And then I ask them what they do when their account gets shut (e.g. because they are selling something bad on Amazon and have the same phone number as the company account?) Or the instance some years ago where GCP closed because someone had wrongly classified image on their drive? Yo…

Another great question is "When did you last try to restore from a backup?" which usually is answered with "It's the built-in tooling, why would we assume it's broken?" or similar. Then fast-forward some months/years, and they try to restore from backups only to realize the backups never actually backed up what they cared about.

In addition to making sure it works, you should make sure you know how to deal with restoring. Sure running a command is easy but what about spinning up new infra? What if it’s corrupted? What if the one person that knows the setup is gone or asleep? Mainly a problem for smaller teams that don’t have the redundancy or resources - they really need to make sure there’s at least docs on how stuff was setup. Reminds me I need to do my yearly checkup too.

Re: "Hetzner decided to cancel our account and terminate all servers"

#172

Post by thesh4d0w »

9cb14c1ec0 wrote: post #93
lakomen wrote: post #79

Earlier quoted context omitted.

Colo is a lot more expensive than some dedi or VM somewhere. Can you provide me with a 12 core 32gb ecc 2TB ssd for 33eur/m? I doubt it.

You have to compare the actually performance of the dedi or VM. Cheap dedis and VMs are usually old, cheap hardware with relatively bad performance. I'm running a 20 core, 96 GB ram, 8TB colocated server for $55/month.

Where are you getting colo space that cheap? I'm moving a non-profit off OVH and onto dedicated and I'm looking at $150/month Canadian for 2u + 400w.

Also, don't forget about hardware acquisition costs, upgrades over time, replacement hardware and downtime due to outages, etc.

Re: "Hetzner decided to cancel our account and terminate all servers"

#173

Post by gruez »

tracerbulletx wrote: post #156
KingOfCoders wrote: post #19

Whenever I ask a CTO if they have a backup (or plan-B) they say we're on AWS, we backup there and they will never go down as a company. And then I ask them what they do when their account gets shut (e.g. because they are selling something bad on Amazon and have the same phone number as the company account?) Or the instance some years ago where GCP closed because someone had wrongly classified image on their drive? Yo…

There should just be a legal duty placed on cloud providers to not do this. Nobody would expect you to hold a second redundant commercial lease for your offices or retail location.

Shouldn't this be covered under standard tort law?

Re: "Hetzner decided to cancel our account and terminate all servers"

#174

Post by bazzargh »

diggan wrote: post #22
KingOfCoders wrote: post #19

Whenever I ask a CTO if they have a backup (or plan-B) they say we're on AWS, we backup there and they will never go down as a company. And then I ask them what they do when their account gets shut (e.g. because they are selling something bad on Amazon and have the same phone number as the company account?) Or the instance some years ago where GCP closed because someone had wrongly classified image on their drive? Yo…

Another great question is "When did you last try to restore from a backup?" which usually is answered with "It's the built-in tooling, why would we assume it's broken?" or similar. Then fast-forward some months/years, and they try to restore from backups only to realize the backups never actually backed up what they cared about.

A fun one like that, a few years back we had some code using dynamodb that used the automatic point-in-time backups. I asked if it had been tested, need you guess the reply?

Of course it turns out that the restore can only happen to a _new database name_ not the original, and the code had in multiple places hardcoded the assumption of what the db was called.

So restoring also involved patching the code and rolling that out; you can't "roll back" because to roll back the db the code must roll forward.

Re: "Hetzner decided to cancel our account and terminate all servers"

#175

Post by Tainnor »

Cumpiler69 wrote: post #102
snehk wrote: post #101

Earlier quoted context omitted.

> [...] so customers who got screwed need to sue and win individually for the same issue which is favorable for the companies doing the screwing as without the precedent of common law This is factually false. > (some) Rental agreements, internet, telco and gym memberships are my favorite infamous examples. They're almost universally regarded as anti-consumer, with tonnes of sketchy clauses, but German lawmakers do no…

>Add that to the fact that any clause is worthless if it includes something a reasonable person wouldn't expect. The problem is you always need to sue to get justice for that which means paying for lawyers and consuming time and money plus stress.

That's true in probably every jurisdiction, though? At least in Germany you can often get free legal advice for many things (Verbraucherschutz, Mietrechtsberatung etc.) and there's insurance you can buy that covers your legal fees in case you lose. And legal fees in Germany are typically not exorbitant.

(Also in some cases, it's the other way around. If your landlord wants to increase the rent it's on them to sue you if they have a valid case.)

Re: "Hetzner decided to cancel our account and terminate all servers"

#176

Post by Tainnor »

Cumpiler69 wrote: post #76
raverbashing wrote: post #64

Earlier quoted context omitted.

Honestly as much as we like to hammer on EU (lack of) companies, one potential improvement point is customer service German companies are awful at customer service. Even within the EU

>German companies are awful at customer service. True also from my experience. I've noted several potential reasons why that is from my time in Germany. Government provided customer protection laws are quite lax and disputes tricky to win and don't represent a big enough deterrent for the scammers when they're just a slap on the wrist and therefore part of the cost of doing business. Sure, you can get sued and you lo…

> partly because of legal reasons where a company can sue your or at least send you scary legal letters for libel if you damage their brand online like that in Germany. Or at lest, the company can simply demand the social media platform take down the offending posts, and by German law they have to comply which the likes of Google/Meta will comply automatically without any arbitration.

I had Google take down my (negative but factual) review of a restaurant because of apparent "libel". There was basically no recourse (except "you can file a complaint but we'll probably ignore it"). I guess that explains why there are so many bad top rated restaurants.

Re: "Hetzner decided to cancel our account and terminate all servers"

#177

Post by goatking »

unixhero wrote: post #67

Hetzner froze my account because I owed them 0.02 EURO's. It was not possible to pay it with a VISA credit or VISA debit, nor Amex card. They required me to wire transfer the money. However my bank does not allow the wiring of a 0.02 EURO amount, as the amount is too low. Out of pure spite I built my own data center.

How did you build your own data center? Is it in your house, or renting a place somewhere? How much did it cost?

I would be curious about any details you can share.

Re: "Hetzner decided to cancel our account and terminate all servers"

#178

Post by hmmm-i-wonder »

KingOfCoders wrote: post #19

Whenever I ask a CTO if they have a backup (or plan-B) they say we're on AWS, we backup there and they will never go down as a company. And then I ask them what they do when their account gets shut (e.g. because they are selling something bad on Amazon and have the same phone number as the company account?) Or the instance some years ago where GCP closed because someone had wrongly classified image on their drive? Yo…

They've gone the route of multiple AWS accounts in my company to avoid the issue they introduced with horrible planning.

First they wanted us out of on-premise, and told us costs wouldn't matter.

Then they wanted us to be 'cloud agnostic', but when given deadlines changed to 'get it working in AWS ASAP, doesn't matter the tech debt'

Now they're freaking out about AWS costs, and we're back to juggling 'cloud agnostic' and 'reduce cost to serve in all clouds' priorities on top of features and maintenance, both of which are 10x slower due to tech debt and the plethora of bugs.

I really need to find a new job soon. Its insane how badly the execs and upper management are running this company. Every day is a knee jerk reaction from someone so detached from the reality of things or with so little understanding how it works, they do nothing but add process problems that barely address the issues they think they're solving.

Re: "Hetzner decided to cancel our account and terminate all servers"

#179

Post by jimjimwii »

n144q wrote: post #148
BoredPositron wrote: post #26

Earlier quoted context omitted.

No response usually means it's a legal case.

iirc Hetzner almost never responds to these posts. On HN or reddit.

Their loss. I'm not touching them with a ten foot pole unless they acknowledge what went wrong and what they will do to make sure they don't fuck up like this again.

Re: "Hetzner decided to cancel our account and terminate all servers"

#180

Post by sourcecodeplz »

amyames wrote: post #120

For anyone else who needs to hear this, Hi, I don’t have a mastodon to reply directly to you. But i have had some issues with content being taken down by VPS providers as well. What I’ve found works well is to use a VPS provider that the public is unaware of. And for some time I had used OVH based on the unlimited bandwidth and the reasoning that Wikipedia and Julian assange (who have far more enemies than I ever wil…

This is really great advice not just for this situation but in general really. For the proxies/what goes in front, I recommend cloudflare workers.
Post reply on HN