"As an experiment, I decided to migrate two hosts (each with about 10 VMs) of a client — where I had full control—without telling them, over a weekend." And that's where I draw the line. Abusing the trust of your customers is an absolute no-no in my book.
Why and how we’re migrating many of our servers from Linux to the BSDs
61–70 of 237 posts
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#62Earlier quoted context omitted.
ZFS has been plenty battle tested on FreeBSD.
Not hardly, and not in the way you think. They replaced their arguably purer ZFS port to replace it with ZoL. As such, it's nowhere near as tested and proven as existing solutions like ext4 and xfs Redhat has deployed to millions of machines for decades. ZFS has too many religious fanboys who hype it without considering that boring and reliable are less risky than betting on code that hasn't had nearly the same scale…
I am well aware of that change.
What specific problems are there with the ZFS implementation on FreeBSD? You claim it is not battle tested, I find it to be rock solid..
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#63Earlier quoted context omitted.
Seems reasonable to me: ... This site and this directory were driving up server load averages, which causes instability for all users on this server. Our block on this site appears to have blocked over 150,000 connections in about half an hour. This is simply more than is appropriate for a shared server. You may want to review your domain's traffic logs to see what kind of traffic this site is getting. The logs for t…
Sure, it's reasonable, if your desired outcome is to lose a customer by a) demonstrating a lack of burst or headroom capacity and/or graceful degradation/priority scheduling capability, and b) being a dick about it. Alternatively one may intervene with a positive approach and framing viz. "we have temporarily [provisioned additional capacity / relocated your storage to another spindle / mirrored your content to a les…
depends on what product/service plan they had. a 5$ per month is not worth jumping through the hoops, because usually an individual solution is needed, but this doesn't scale. if the site is important to the owner, they need a more fitting service plan with higher headroom.
that being said, a good provider at least tries to filter out what could be a (d)dos
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#64I wish he could write up a bit about XFS failure he had. Im using it from many many years and there is no issues at all.
It hard to know the timeline with his data loss, but I am assuming it was a long time ago. XFS is originally from SGI Irix and was designed to run on higher end hardware. SGI donated it to Linux in 1999 and it carried a lot of its assumptions over. For example on SGI boxes you had "hardware raid" with cache, which essentially is a sort of embedded computer with it's own memory. That cache had a battery backup so that…
> For example on SGI boxes you had "hardware raid" with cache, which essentially is a sort of embedded computer with it's own memory. That cache had a battery backup so that if the machine had a crash or sudden power loss the hardware raid would live on long enough to finish its writes. SGI had tight control over the type of hardware you could use and it was usually good quality stuff.
Most of the SGI machine I've used of various sizes did not have hardware raid. In my experience, you were more likely to run into hardware raid on a PC than on traditional SGI or Sun servers (I don't have much experience with AIX or HP-UX), unless the unix server was in a SAN environment.
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#65Earlier quoted context omitted.
Not hardly, and not in the way you think. They replaced their arguably purer ZFS port to replace it with ZoL. As such, it's nowhere near as tested and proven as existing solutions like ext4 and xfs Redhat has deployed to millions of machines for decades. ZFS has too many religious fanboys who hype it without considering that boring and reliable are less risky than betting on code that hasn't had nearly the same scale…
I am well aware of that change. What specific problems are there with the ZFS implementation on FreeBSD? You claim it is not battle tested, I find it to be rock solid..
> You claim it is not battle tested, I find it to be rock solid
Those two things are not mutually exclusive.
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#66Earlier quoted context omitted.
Ah, pair. Reminds me of this gem: http://www.archub.org/ycoutage.txt
Seems reasonable to me: ... This site and this directory were driving up server load averages, which causes instability for all users on this server. Our block on this site appears to have blocked over 150,000 connections in about half an hour. This is simply more than is appropriate for a shared server. You may want to review your domain's traffic logs to see what kind of traffic this site is getting. The logs for t…
That's not even remotely reasonable.
They should have been monitoring load and notified the customer before it got to the point that they were worried about stability.
Shutting down a customer's site without warning when you could have notified them of a problem in the near future... well, that's a great way to lose a customer. I really hope YC promptly moved to another provider after that incident.
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#67"As an experiment, I decided to migrate two hosts (each with about 10 VMs) of a client — where I had full control—without telling them, over a weekend." And that's where I draw the line. Abusing the trust of your customers is an absolute no-no in my book.
Not an abuse at all. I've a contract with those clients, and I can move the VMs, change the services, etc. freely as long as it doesn't cost more than the amount we've previously set.
Otherwise, I'd never dare to do something like that.
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#68> “If nothing is working, what am I paying you for? If everything’s working, what am I paying you for?” Bloke is not acquainted with Keynesian economics. https://www.youtube.com/watch?v=9OhIdDNtSv0 https://www.youtube.com/watch?v=NO_tTnpof_o All a man needs is food in his stomach and a place to rest at the end of the day. Everything else is vanity What proportion of global GDP is dedicated to fulfilling our basic mat…
The problem with the IT crowd is they think they are ahead of the optimization curve, and since everyone does, nobody is.
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#69All is fine and dandy and BSDs can solve many use cases. Unfortunately for the solution we are working on, which imply many microservices we need Kubernetes and no BSD equivalent to Kubernetes exists.
Same here, otherwise I'd be considering the BSDs.
Re: Why and how we’re migrating many of our servers from Linux to the BSDs
#70I wish he could write up a bit about XFS failure he had. Im using it from many many years and there is no issues at all.
I'm interested too. I'm using XFS only, and have for many years. On my own boxes, but my company also uses XFS for all the data on customer computers. We did extensive testing many years back, and XFS was the only filesystem at the time which gave a linear, constantly very high performance when writing and reading huge amounts of data (real-time data, dips in performance is a 100% no-no), and also not degrading when having huge numbers of files. We've never had a customer lose data due to XFS problems, and at this point I can't imagine how much data that would be, except that it's astronomical.
When that's said, we had routine XFS losses on SGI boxes. That was a very well known scenario: Write constantly to a one-page text file, say, every few seconds, then power cycle the machine. The file would be empty afterwards. This doesn't happen on Linux, I vaguely recall discussing this with someone some years ago (maybe here on HN) and something was changed at some point, maybe when SGI migrated XFS to Linux, or shortly after.