U
Why and how we’re migrating many of our servers from Linux to the BSDs
131–140 of 237 posts
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#132Earlier quoted context omitted.
Agree with this on all points. We went from a virtualized server model to managed Kubernetes and costs have escalated considerably. The additional complexity and maintenance overheads of Kubernetes are not trivial and required additional staff hires just to keep things ticking. I think the cost so far from moving from two cages in separate datacentres running blades to AWS is approximately a 6x multiplier including s…
That's because you've moved to AWS which is an expensive cloud. You could have easily run Kubernetes on your virtualised servers.
Define easily?
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#133Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#134> The largest failure was with btrfs — after a reboot, a 50 TB filesystem (in mirror, for backups) simply stopped working. No more mounting possible. Data was lost, but I had further backups. The client was informed and understood the situation. Within a few days, the server was rebuilt from scratch on FreeBSD with ZFS — since then, I haven’t lost a single bit. As someone who admins a lot of btrfs, it seems very unli…
> As someone who admins a lot of btrfs, it seems very unlikely that this was unrecoverable.
As someone who used it all day every day in my day job for 4 years, I find it 100% believable.
I am not saying you're wrong: I'm saying, experiences differ widely, and your patterns of use are not be universal.
It's the single most unreliable untrustworthy filesystem I've used in the 21st century.
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#135I've recently discovered systemd-nspawn which is an alternative to LXC, builtin and integrated into systemd. Much lighter than full VMs and it's quite similar to Solaris Zones and FreeBSD jails. One way to use it is to extract an OCI (Docker) image to a path, that way you can reuse the container tooling provided by Docker, Podman et al. I've barely touched the BSDs and it's been a few years since I last used Solaris…
Thanks for this! I have been using LXC/LXD for a long time and never knew about systemd-nspawn. Time to go learn something new!
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#136Earlier quoted context omitted.
Facebook runs their entire stack using Btrfs [0]. I would encourage anyone who is stuck in the "oh btrfs is so buggy and loses data" mindset (not helped by articles like this [1] that play off btrfs as some half-baked contraption, when it's really btrfs raid that needs a LOT more time to bake) to look into things and realize that large companies (OpenSuse, Redhat, Faceboook) have poured a lot of time to get it to wor…
I am stuck in the btrfs-is-buggy mindset precisely because it managed to lose my root partition on a single disk machine. It might also have raid problems, but not exclusively.
Me too. Repeatedly, at least once a year, on 3 different machines.
The cause? Filling up the filesystem. Why? Because of OS snapshots.
(Aside: why can they fill it? Because it doesn't give a straight answer to `df -h`. Why not? Because of snapshots.)
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#137Earlier quoted context omitted.
ZFS on Linux is not really usable for most users because every kernel update can break your ZFS compatibility. Meaning unless you want to put in the time to manually test every kernel update and ensure your kernel version stays in-sync with OpenZFS you can very likely end up with an unbootable system.
This is one of the main reasons Void Linux is "stuck" on kernel 6.6.
You mean apart from 6.6 being the current latest longterm kernel?
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#138I've become a fairly loyal OpenBSD user in the last 3-4 years. The base OpenBSD load includes a substantial amount of network capabilities, and is cleanly implemented. It's almost too cleanly implemented, to the point of making me feel sort of guilty when I start to clutter up an install with a bunch of packages...
If my needs for storage were more complicated, I would probably use FreeBSD ZFS, but UFS suffices for my rather modest needs.
I use OpenBSD for desktop, web and mail services. There are some limitations, but none that are serious enough to warrant dealing with running another BSD, or Linux distribution.
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#139Earlier quoted context omitted.
For a long time they didn't have any. I recall from mid 90s version 4+? They were still trying to port.. or was it early 2000s lol either way it's ancient history
Yeah I started around 4.something until 6 I believe. After that it was all Debian. My lesson was to just use what moet people use. But then again, I always used PostgreSQL even though most were on MySQL
> My lesson was to just use what moet people use.
Funny, my lesson was nearly the opposite: Don't try to use something that's popular but that meets my needs the best.
I used Mac for a long time but i got allergic to opinionated software. Tried Ubuntu for a while but hated the "not invented here" and commercial motives (eg even distributing small console stuff in snaps). I'm now very happy with FreeBSD on my primary desktop.
But of course my needs and things I care about are different.
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#140Earlier quoted context omitted.
Facebook runs their entire stack using Btrfs [0]. I would encourage anyone who is stuck in the "oh btrfs is so buggy and loses data" mindset (not helped by articles like this [1] that play off btrfs as some half-baked contraption, when it's really btrfs raid that needs a LOT more time to bake) to look into things and realize that large companies (OpenSuse, Redhat, Faceboook) have poured a lot of time to get it to wor…
Meta does a lot of things that don't scale for reliable/trustworthy systems and aren't suitable for all use-cases. (I also used to work there too.) ZFS is only reliable where it was battled-tested: on Solaris. ZoL has been infinitely tinkered with and smashed up that it's nothing like running a Thumper as a NAS. XFS + mdadm on Linux is, without a doubt, far more reliable than ZoL. Ask me how I know. I have the scars…
ZFS on Linux is absolutely fine in high-performance and critical computing applications.
I also owned a Thumper and Thor running Solaris in 2009. Much prefer Linux and the hardware solutions today.