[flagged]
Kiwix does not operate Mastodon servers.
Deep. Thanks, my brain apparently does not work while I’m fasting.
"Hetzner decided to cancel our account and terminate all servers"
121–130 of 401 posts
Re: "Hetzner decided to cancel our account and terminate all servers"
#122Whenever I ask a CTO if they have a backup (or plan-B) they say we're on AWS, we backup there and they will never go down as a company. And then I ask them what they do when their account gets shut (e.g. because they are selling something bad on Amazon and have the same phone number as the company account?) Or the instance some years ago where GCP closed because someone had wrongly classified image on their drive? Yo…
Another great question is "When did you last try to restore from a backup?" which usually is answered with "It's the built-in tooling, why would we assume it's broken?" or similar. Then fast-forward some months/years, and they try to restore from backups only to realize the backups never actually backed up what they cared about.
aws-cli will sync your s3 buckets to a local system.
I’m doing that to linux, and then the Linux box is furthermore backed up with nakivo.
Not my favorite but the price was okay and I can run the whole director on Linux, unlike all their other competitors. [veeam’s next major release 13 or 14 should do this in the next year or so too.]
While nakivo backs up s3 buckets, NFS shares, and local file servers… to your point, I don’t trust it (or any other backup software I can’t extract and unpack the resulting backup by hand) as far as I can throw it. So I rsync or mirror it to a local Linux box with aws-cli and then back THAT up.
I think you can do all this with windows stuff too but I don’t know it that well
Additionally you can take servers that are linux vps’es and do the reverse: mirror THEIR content to an s3 bucket.
You can also run minio open source/free on your fileserver and set up s3 to s3 sync. Cloudflare for example will ingest and replicate your minio server automatically and you can firewall it all off to their address ranges. It’s not free but it actually prices out favorably compared to veeam and nakivo if that’s all you need backed up.
Re: "Hetzner decided to cancel our account and terminate all servers"
#123Earlier quoted context omitted.
What if you don't want to host your stuff in the same jurisdiction where you live because you don't trust your government?
There are colocation datacenters all over the world.
Sure, but if something goes wrong with your colocated server, you're supposed to fix it yourself, aren't you? So it feels kinda important that the datacenter is close enough so you could get there quickly on a short notice. I'm imagining having to fly several hours and cross borders just to replace a failed hard drive, all while your server is down.
Re: "Hetzner decided to cancel our account and terminate all servers"
#124Re: "Hetzner decided to cancel our account and terminate all servers"
#125Would be good to have a fall-back solution, is there something similar in dedicated server price in the EU as Hetzner? Or does no on else come close?
Re: "Hetzner decided to cancel our account and terminate all servers"
#126Earlier quoted context omitted.
Or, in one Mastodon instances case recently, because Hetzner saw that users could upload their own images Wait, what? Yikes. I'm planning a project like that. Do you have a link to more information?
https://woem.men/notes/9r86xd69cu89052m Also shoutout to Cloudflare for showing off what a diverse company they are in this one /s
That wasn't even the one I was thinking of; sounds like this has been happening a lot.
Re: "Hetzner decided to cancel our account and terminate all servers"
#127Been a Hetzner customer for years and have considered using them for a new business project of mine. Will reconsider it partly after reading this. At least use a separate provider for backups so I can quickly recover, just in case.
Seeing it happen to a reputable project such as Kiwix [0] definitely damages my perception of Hetzner. I've read numerous complains on Reddit a few months ago but they mostly boiled down to breaching the ToS in obvious ways. Still, not giving a heads up before cancelling a service and no option to recover data is just bad business practice.
[0] (I've deployed Pi's with Kiwix in remote areas in Africa, it's an amazing project)
Re: "Hetzner decided to cancel our account and terminate all servers"
#128We need to boycott hetzner.
Re: "Hetzner decided to cancel our account and terminate all servers"
#129Earlier quoted context omitted.
Another great question is "When did you last try to restore from a backup?" which usually is answered with "It's the built-in tooling, why would we assume it's broken?" or similar. Then fast-forward some months/years, and they try to restore from backups only to realize the backups never actually backed up what they cared about.
Agreed, if you haven't tested your backups recently (daily, automatic best), you don't have backups. Several of my clients (CTO Coaching) had problems in the past because they restored backups and where finding they were not complete (for various reasons).
daily test restore is infeasible for anything but toy projects.
You should periodically test your restore procedures, but its incredibly costly and time consuming for sizeable platforms. Its just not that easy to restore a 10+TB backup for example, and thats a _tiny_ backup size for a b2c product.
they can easily go into the hundreds of TB, depending on your platform.
and i might add: i vividly remember gitlabs article how they have had automated backups and test restores for years, but when they actually needed them... it turned out some data wasn't part of it after all. just because youre testing your restore procedure doesnt mean you've actually accomplished anything.
Re: "Hetzner decided to cancel our account and terminate all servers"
#130Hetzner did something similar a couple of years ago, suddenly disabling 1000 Solana validators that were using their service: https://www.theblock.co/post/182283/1000-solana-validators-g...
That doesn't sound very similar at all. Their Terms and Conditions specifically say they don't allow cryptocurrency mining or similar, so hardly surprising that they shut down something like Solana validators.
Solana validators do not perform cryptocurrency mining.