Earlier quoted context omitted.
Yes. $30-$50 each. Multiply that by 60 or 90 and multiply that by (however many you put in a rack). That adds up to five figure premiums on drives we're going to burn in anyway . We know how to read an HCL - it's not rocket science.
Ok 60X50 is 3K even with extra 3K Supermicro is still a lot cheeper
832 TB – ZFS on Linux
151–160 of 165 posts
Re: 832 TB – ZFS on Linux
#152Earlier quoted context omitted.
But obviously Canonical's lawyers don't think shipping it separately is the same as shipping it in the kernel, because if they did they would not ship it separately to begin with.
Granted. But Canonical's lawyers are not canonical. There's legal disagreement, and saying "Ubuntu did it" doesn't make those problems disappear. (Sadly)
I -have- seen a lot of argument online, opinions and otherwise that disagree with this, but not anything from a lawyer - mainly "philosophical" or opining on the "intent" of the GPL, rather than "based on clauses x, y, and z, this is impermissible".
Which is not to say, if something of an opinion exists in response to Canonical's release of ZFS, I would love to see it - a legal opinion, not armchair lawyering or opinionizing.
Re: 832 TB – ZFS on Linux
#153Currently each server has 63 drives (4TB HGST NL-SAS) with 1 hot spare, configured as RAIDZ.
Right now there is 200TB of usable storage, we initially started with 29TB and have been expanding as needed when it hits about 79%, I buy 18 drives roughly every 6-8 months, 9 drives per server and expand the pool.
To say that we never had issues is lying, we did have some major issues when upgrading from versions, but this was early on, now it is a rock solid storage system.
Although there is less than 300 active users connecting to the primary server, there is a lot of very important pre & post production high dev videos.
Reboot with 63 drives is around 10 minutes or less.
Resilvering could take 24-48 hours, depending on load, depending on how much data the failed drive contained.
Performance has been great, reliability has been great, support has been great.
Sadly IX Systems can no longer provide support after the end of this year, they've extended support beyond the expected lifetime of the hardware.
Re: 832 TB – ZFS on Linux
#154Earlier quoted context omitted.
Been running an 8 drive array for nearly 5 years now myself and never once had an issue through 3 PCs and 3 different operating systems. What was your issue?
With utmost respect: this isn't a super valuable data point. A "production deployment" with lots of users and/or workloads will see issues you will never encounter in a moderate setting. FWIW, I use FreeNAS in a similar small but diverse setting with nary a problem, but when I sat it up for a 30+ organization, issue arose that I hadn't expected (not data loss, but usability and performance issues).
Re: 832 TB – ZFS on Linux
#155Re: 832 TB – ZFS on Linux
#156There are a couple needful tweaks to this BOM for anyone wanting to follow this.. Only populate one CPU socket. Zone allocation between two NUMA nodes is kind of hard, especially since Ubuntu 16.04 zfs is pre- OpenZFS ABD where memory fragmentation is reality. I would recommend better NICs like a Chelsio T5 or T6. Aside from better drivers and a responsive vendor, you can experiment with some of the iscsi offloads or…
Why FreeBSD?
And for self-support, the freebsd-scsi and freebsd-net mailing lists will get you in touch right away with the people that understand the firmware, protocols, drivers, and systems engineering. You will probably get a response from a kernel and operations aware developers from Dell Isilon, SpectraLogic, or Netflix within the day that commit the code you are using. You probably wont transcend customer support at other organizations in that timeframe.
Re: 832 TB – ZFS on Linux
#157Can I just ask. Why not use FreeBSD?
Re: 832 TB – ZFS on Linux
#158Earlier quoted context omitted.
I disagree with the use of the word "deprecating", and everyone has been parroting around the news as though Red Hat announced that btrfs causes machines to catch on fire. Red Hat never provided enterprise support for btrfs, it was a technical preview that didn't pan out to become fully supported as part of their distribution. It's barely a story (there are plenty of other filesystems Red Hat doesn't support), but it…
> I disagree with the use of the word "deprecating" Then you disagree (literally) with Red Hat's official statement as a matter of fact (not opinion). From Chapter 53, RHEL 7.4 release notes: "Btrfs has been deprecated ... Red Hat will not be moving Btrfs to a fully supported feature and it will be removed in a future major release of Red Hat Enterprise Linux."[0] Pretty definitive. [0] https://access.redhat.com/docu…
Re: 832 TB – ZFS on Linux
#159At that scale something like Ceph would be more reasonable. Just because ZFS can handle those filesystem sizes doesn't necessarily mean that it's the best tool for the job. There's a reason why all big players like Google, Amazon and Facebook go for the horizontal scaling approach.
Re: 832 TB – ZFS on Linux
#160Earlier quoted context omitted.
That's not really a fair statement. First of all, Linux started as Linus just reimplementing the Unix semantics, so edge cases and subtle semantics should be expected to be different. Secondly, most Unix-based OSes are barely compatible in their facilities. Sure, you have DTrace (which only macOS, illumos and FreeBSD have) and ZFS (which only illumos and FreeBSD have) but the rest is similarly incompatible. Solairs/i…
Sun's event ports are so similar to kqueue cantrill has said they should have just adopted kqueue. And there are not different views on zones and jails. They are the same thing. Sun just took the idea of jails and flushed them out further adding a separate network stack for each zone, which jails now also have. But they operate on the same principle and ideas. Jails was bare bones at inception. Jails shared their IP…
* You were discussing porting a Linux application, not a kernel feature like DTrace, ZFS, Zones, kqueue, etc. Obviously porting a kernel feature between two OSes that share a kernel history is going to be easier than porting to an entirely different kernel. It's almost tautological. Porting an application has more to do with whether the syscall/libc interfaces are compatible and if you take kqueue/eventports as an example you still need to do standard porting work. glibc provides BSD-like interfaces so it's not like you have to switch away from bzero or whatever -- that's not the hard part of porting code.
* You specifically stated that Linux is "off in the weeds", "doing their own poor re-implementations of tech others have already done". Ignoring how disrespectful that is, your response to me saying "the whole Unix family re-implements each others ideas all the time -- DTrace and ZFS are the exception and only three members of the family use them" isn't helping your original point.
Also this whole section is just a non-sequitur:
> And there are not different views on zones and jails. They are the same thing. [Long description of how they are different and were developed separately.] [Random aside about Linux containers and how they're a mess.] So jail and zones are similar.
I am aware of the similarities and differences between Jails and Zones, and I'm also very painfully familiar with Linux containers. Not sure why you're bringing them up in a discussion about porting applications between different Unix-like operating systems. Sounds like you just have an axe to grind.