Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why and how we’re migrating many of our servers from Linux to the BSDs

it-notes.dragas.net

Post reply on HN
Print view View original

201–210 of 237 posts

Re: Why and how we’re migrating many of our servers from Linux to the BSDs

#201

Post by kev009 »

knowitnone wrote: post #162
kev009 wrote: post #79

I use Linux, FreeBSD, NetBSD, and OpenBSD all for fun, learning, and profit(the first two). At the very least it is nice to make acquaintance with at least one BSD because it will probably expand your knowledge on Linux in ways you wont be able to anticipate. For example, FreeBSD got me into kernel development, full system debugging, network stack development, driver development, and understanding how the whole kit f…

I see the reason for dual stack but I would rather focus my efforts into securing one OS. If you buy into that, why not 3 or 4 different OSes?

It's an engineering tradeoff and market adaptation.

If you are a typical SaaS provider the complexity may be beyond your capabilities and budgets.. allegory to a local delivery business choosing to build a long term relationship with a single vehicle manufacturer and dealer.

If you are a high stakes service provider, you need to start thinking about how to get out from being controlled by a single vendor and market flux and plot your own destiny.. allegory to a national shipping carrier sourcing vehicles from multiple manufacturers and developing long term relationships with them to refine the platform.

Re: Why and how we’re migrating many of our servers from Linux to the BSDs

#202

Post by inopinatus »

stackskipton wrote: post #148
taskforcegemini wrote: post #63

Earlier quoted context omitted.

depends on what product/service plan they had. a 5$ per month is not worth jumping through the hoops, because usually an individual solution is needed, but this doesn't scale. if the site is important to the owner, they need a more fitting service plan with higher headroom. that being said, a good provider at least tries to filter out what could be a (d)dos

In 2010, those tools were not easily available outside massive Enterprise solutions like Akamai. Cloudflare just launched in 2009 and on-premise hardware was generally cost prohibitive.

I was building ISPs from the 1990s. The idea that after decades of innovation and field experience, tools for managing bursty/high-growth customers were somehow "not easily available", or that hardware be some kind of unicorn resource, just smells to me like rank incompetence. In this case coupled to an equally questionable customer retention failure.

Re: Why and how we’re migrating many of our servers from Linux to the BSDs

#203

Post by traceroute66 »

SubjectToChange wrote: post #195
traceroute66 wrote: post #178

Earlier quoted context omitted.

> Facebook (well, Meta, I guess) is famously a big user and developer of btrfs. It seems to work just fine for them I really, really, really wish people would STOP with the whole "it works for $SilconValleyCorp so it must work for me" or "$SiliconValleyCorp does it, so I must". It only leads to disappointment in the case of the former and wholly un-necessary over-engineering in the case of the latter. (a) You do not…

Look, I simply highlighted a major user of btrfs. Sorry if you have some complex emotions about them. But for some reason, I doubt you'd say the same thing when someone mentions Netflix using FreeBSD. >(a) You do not know how or where Facebook use BTRFS Their engineering team has posted a few of their use cases. >(c) Facebook probably employ the guy who invented BTRFS and an army of kernel developers on top of that .…

> But for some reason, I doubt you'd say the same thing when someone mentions Netflix using FreeBSD.

Actually, I would. Not because I'm a BSD hater, because we actually use a lot of BSD at $work.

But instead because I reckon I could safely win a bet with you that Netflix do not use the vanilla version of FreeBSD.

Most people I know would agree with me that the secret sauce will forever stay secret.

Sure, without a doubt Netflix contribute stuff back to FreeBSD. But I betcha it's not ALL the stuff. :)

The same goes for other famous FreeBSD users, e.g. Juniper Networks.

I'm happy to recommend FreeBSD to people, but if they're looking for Netflix or Juniper level network performance, they'll need to know they'll have to do the donkey work themselves because there's not a cats chance in hell they'll magically get it "out of the box".

Re: Why and how we’re migrating many of our servers from Linux to the BSDs

#204

Post by E39M5S62 »

binkHN wrote: post #44
jsiepkes wrote: post #38

Earlier quoted context omitted.

ZFS on Linux is not really usable for most users because every kernel update can break your ZFS compatibility. Meaning unless you want to put in the time to manually test every kernel update and ensure your kernel version stays in-sync with OpenZFS you can very likely end up with an unbootable system.

This is one of the main reasons Void Linux is "stuck" on kernel 6.6.

The 'linux' package on Void is just a meta package. Install whatever kernel series you want. I'm running 6.10.11, with ZFS 2.2.6 on my Void workstation.

Re: Why and how we’re migrating many of our servers from Linux to the BSDs

#205

Post by riiii »

dang wrote: post #28
riiii wrote: post #12

[flagged]

Please don't do this here. Not that we don't appreciate a good old-fashioned flame, but that the long-term costs outweigh the benefits, and we want this site to survive long-term.

That was sarcasm not flame. But it's always 60/60 chance sarcasm will just blow up.

Re: Why and how we’re migrating many of our servers from Linux to the BSDs

#206

Post by barkingcat »

graemep wrote: post #108
kev009 wrote: post #79

I use Linux, FreeBSD, NetBSD, and OpenBSD all for fun, learning, and profit(the first two). At the very least it is nice to make acquaintance with at least one BSD because it will probably expand your knowledge on Linux in ways you wont be able to anticipate. For example, FreeBSD got me into kernel development, full system debugging, network stack development, driver development, and understanding how the whole kit f…

> tend to use FreeBSD because in many ways it is simpler and less surprising, especially when accounting for the passage of time. ifconfig is still ifconfig, and it works great. rc.d is all I need for my own stuff. That sounds very appealing to me. I have to keep a small number of servers running, but its not my main focus and I would like to spend as little time on it as possible. I have started using Alpine Linux f…

I've run the same freebsd system for the last 15 years just hosting a small site for some of my friends from back in university.

I've migrated it through system upgrades and security fixes, but nothing else needed to change. usual uptime is about 3 years between major release updates.

freebsd is an awesome server platform.

Re: Why and how we’re migrating many of our servers from Linux to the BSDs

#207

Post by ab71e5 »

kev009 wrote: post #199
ab71e5 wrote: post #106

Earlier quoted context omitted.

What sort of projects did you do to get into network stack and driver development on BSD?

I worked at a large service provider and had to fix bugs and improve performance in TCP/IP and Intel Ethernet drivers. I ended up really enjoying it so it became a hobby and consulting gig as well.

Awesome, you are kind of describing my dream job. Any particular hobby projects you are willing to share? I'm working on something myself trying to use eBPF/XDP offloading for a particular protocol

Re: Why and how we’re migrating many of our servers from Linux to the BSDs

#208

Post by kev009 »

traceroute66 wrote: post #203
SubjectToChange wrote: post #195

Earlier quoted context omitted.

Look, I simply highlighted a major user of btrfs. Sorry if you have some complex emotions about them. But for some reason, I doubt you'd say the same thing when someone mentions Netflix using FreeBSD. >(a) You do not know how or where Facebook use BTRFS Their engineering team has posted a few of their use cases. >(c) Facebook probably employ the guy who invented BTRFS and an army of kernel developers on top of that .…

> But for some reason, I doubt you'd say the same thing when someone mentions Netflix using FreeBSD. Actually, I would. Not because I'm a BSD hater, because we actually use a lot of BSD at $work. But instead because I reckon I could safely win a bet with you that Netflix do not use the vanilla version of FreeBSD. Most people I know would agree with me that the secret sauce will forever stay secret. Sure, without a do…

Bet accepted.

Netflix runs -CURRENT and they have been very vocal about their approach to both developing and running a vanilla FreeBSD tree, i.e. https://freebsdfoundation.org/netflix-case-study/

In my personal experience, if you reach out to them, they will likely help you where it relates to their work and expertise including collaborating on works in progress that are not ready for main. You do have to configure and use appropriate software to get the same numbers as they, i.e. sendfile and some sysctls but any local changes they have would not be material to posting similar performance numbers.

How do I collect my winnings :)

Re: Why and how we’re migrating many of our servers from Linux to the BSDs

#209

Post by herzzolf »

thoroughburro wrote: post #130

> The largest failure was with btrfs — after a reboot, a 50 TB filesystem (in mirror, for backups) simply stopped working. No more mounting possible. Data was lost, but I had further backups. The client was informed and understood the situation. Within a few days, the server was rebuilt from scratch on FreeBSD with ZFS — since then, I haven’t lost a single bit. As someone who admins a lot of btrfs, it seems very unli…

FWIW, this wasn't always the case. I recall that BTRFS reliability was much different, say, 10–15 years ago. The post touched those ancient times as well, so that isn't that much of a stretch.

Around that time, SLES made btrfs their default filesystem. It caused so many problems for users that they reversed that decision almost immediately.

Re: Why and how we’re migrating many of our servers from Linux to the BSDs

#210

Post by kev009 »

ab71e5 wrote: post #207
kev009 wrote: post #199

Earlier quoted context omitted.

I worked at a large service provider and had to fix bugs and improve performance in TCP/IP and Intel Ethernet drivers. I ended up really enjoying it so it became a hobby and consulting gig as well.

Awesome, you are kind of describing my dream job. Any particular hobby projects you are willing to share? I'm working on something myself trying to use eBPF/XDP offloading for a particular protocol

Lately I have been enjoying NetBSD on a EdgeRouter-4 (Octeon MIPS). I am trying to port some of the OpenBSD Octeon improvements like the MMC driver and maybe SMP support. On FreeBSD I have been pushing some changes to the 1, 2.5, and 10gbit Intel drivers that will decrease driver overhead a few percent.
Post reply on HN