Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

832 TB – ZFS on Linux

jonkensy.com

Post reply on HN
Print view View original

51–60 of 165 posts

Re: 832 TB – ZFS on Linux

#51

Post by chungy »

ethbro wrote: post #48
rdeckard0104 wrote: post #43

Earlier quoted context omitted.

I disagree. FreeBSD is just as easy to administer as Linux. Solaris I find to be a bit more challenging. I can appreciate your commentary, but I challenge you to prove me wrong. Just for some color I've ran the Gambit of Unix and Linux systems in my career with FreeBSD being one of the easiest and most consistent out of the box technologies. Solaris had already been quite the opposite. I guess you could say AIX is a…

> For every competent FreeBSD or Illumos sysadmin there are 10x equally experienced with Linux. Was parent wrong about that point?

Not exactly, but to any competent administrator, learning FreeBSD based on Linux experience is not that difficult. It's sort of like tasking someone to start working on a program written in Go even if they have only C++ experience. You can cope.

Re: 832 TB – ZFS on Linux

#52

Post by barrkel »

qeternity wrote: post #16
overcast wrote: post #9

Earlier quoted context omitted.

ZFS is a native filesystem to FreeBSD, where as in Linux it is not.

Sure but that alone doesn't mean it's any less suitable on Ubuntu. I'm just trying to understand what the differences are in implementation, reliability, etc.

ZoL uses an emulation layer to make Linux look more like something with Unix ancestry, ancestry that BSD and Solaris share. The biggest difference I believe is in memory management / fs cache integration.

In practice this means that ZoL may require more memory to be stable, or may be less stable depending on configuration if low in memory.

I've run ZFS on Solaris (by way of Nexenta) and Linux since 2008 or so. I haven't seen much reliability difference in practice. I've had fewer hiccups streaming video from Linux though.

Re: 832 TB – ZFS on Linux

#53

Post by X86BSD »

cmurf wrote: post #49
scottLobster wrote: post #38

Earlier quoted context omitted.

RedHat is apparently building something new called Stratis based on XFS, they're shooting for delivery within the next 1-2 years. Supposedly it'll be feature-equivalent to ZFS/Btrfs/LVM. http://www.phoronix.com/scan.php?page=news_item&px=Stratis-R...

The ZFS comment in the document [1] is time inspecific. It's got version 1.0 in 1st half of 2018 and the "rough ZFS feature parity" comment is not until version 3.0, which has no listed time frame. This is going to take lot of work, and not just for the stratis developers but for projects that need to manipulate it. It's asking for a lot of work for bootloader projects to support it, and [1] https://stratis-storage.g…

Filesystems are HARD, see the plethora of FS's on Linux that are dead, dying, or horribly engineered. This seems like such a monumental waste of resources, time, and energy to chase ZFS, which itself is not a stationary target. ZFS is constantly developing new features. I think a lack of ZFS in Linux is hurting it. And that wound will only deepen over time.

Re: 832 TB – ZFS on Linux

#54

Post by hengheng »

rsync wrote: post #7

"I ended up between the Supermicro SSG-6048R-E1CR60L or the SSG-6048R-E1CR90L – the E1CR60L is a 60-bay 4U chassis while the E1CR90L is a 90-bay 4U chassis. This nice part is that no matter which platform you choose Supermicro sells this only as a pre-configured machine – this means that their engineers are going to make sure that the hardware you choose to put in this is all from a known compatibility list. Basicall…

What competitors are there? Their hardware is getting more and more specific, looking at e. g. their four-socket systems or their 2U quad-systems. Nothing that doesn't handle like any other computer, but hard to cross-shop for.

Re: 832 TB – ZFS on Linux

#55

Post by [deleted] »

rsync wrote: post #7

"I ended up between the Supermicro SSG-6048R-E1CR60L or the SSG-6048R-E1CR90L – the E1CR60L is a 60-bay 4U chassis while the E1CR90L is a 90-bay 4U chassis. This nice part is that no matter which platform you choose Supermicro sells this only as a pre-configured machine – this means that their engineers are going to make sure that the hardware you choose to put in this is all from a known compatibility list. Basicall…

[deleted]

Re: 832 TB – ZFS on Linux

#57

Post by gruturo »

zorked wrote: post #40
gruturo wrote: post #35

Earlier quoted context omitted.

ZFS on FreeBSD and on Illumos has been used, abused, tested and stressed all the way to the Moon and beyond. Loads of people with loads of data over loads time. This makes bugs appear, and get addressed, and eventually gives you confidence that nothing nasty remains uncorrected, and it won't eat your data. ZFS on Linux, due to the unfortunate licensing situation, is considerably less tested and thus scary, data-eatin…

On the other hand Linux is far more popular so I wouldn't be surprised if ZFS on Linux already clocks more hours of usage than on BSD.

> On the other hand Linux is far more popular so I wouldn't be surprised if ZFS on Linux already clocks more hours of usage than on BSD.

Yes and no. Few people, even among the Linux crowd, are aware it exists or feel like trusting it with important data sets/jobs. Those few are the most likely to be equally at ease running FreeBSD or Illumos (or Solaris), where the damn thing is known to work really well.

Re: 832 TB – ZFS on Linux

#58

Post by floatboth »

X86BSD wrote: post #10
4ad wrote: post #2

you can run ZFS on Ubuntu [...] You could also build this on Solaris with necessary licensing if you wanted to that route but it’d be more expensive. I find it bewildering the author didn't even consider illumos or FreeBSD, where ZFS is a first class citizen.

Yeah I was like uhhhh what? Illumos and illumos based OS'es and FreeBSD are the only games in town where ZFS is a first class citizen. Ill chalk it up to extreme bias.

Yeah, many people don't consider FreeBSD because of silly reasons… :(

but mentioning Solaris that "requires licensing" and not illumos?? illumos is the new Solaris, the old Solaris devs are working on illumos! Oracle Solaris is irrelevant.

Re: 832 TB – ZFS on Linux

#59

Post by gribbly »

ethbro wrote: post #47
gribbly wrote: post #44

Earlier quoted context omitted.

>One theory is that Canonical has essentially forced Oracle to decide whether it wants to crack down on inclusion of CDDL-licensed code being shipped with Ubuntu's stock kernel Huh? Are Canonical is shipping ZFS in their kernel now ? I thought they just distributed it as a separate module.

There was an article a bit ago on HN where the Free Software Conservancy (?) wrote that legally, shipping a precompiled binary kernel module is the same as having it in the kernel.

But obviously Canonical's lawyers don't think shipping it separately is the same as shipping it in the kernel, because if they did they would not ship it separately to begin with.

Re: 832 TB – ZFS on Linux

#60

Post by luckydude »

cmurf wrote: post #49
scottLobster wrote: post #38

Earlier quoted context omitted.

RedHat is apparently building something new called Stratis based on XFS, they're shooting for delivery within the next 1-2 years. Supposedly it'll be feature-equivalent to ZFS/Btrfs/LVM. http://www.phoronix.com/scan.php?page=news_item&px=Stratis-R...

The ZFS comment in the document [1] is time inspecific. It's got version 1.0 in 1st half of 2018 and the "rough ZFS feature parity" comment is not until version 3.0, which has no listed time frame. This is going to take lot of work, and not just for the stratis developers but for projects that need to manipulate it. It's asking for a lot of work for bootloader projects to support it, and [1] https://stratis-storage.g…

I'm friends with some of the original ZFS guys, Jeff Bonwick was a student of mine at Stanford and I got him to come to Sun, Bill Moore worked for me on BitKeeper. Those guys are seriously studly engineers. I've done file system work at Sun, I don't compare with Bill & Jeff, they are way better.

So yeah, trying to do better than those guys is going to be interesting. I'd like to know who is on the RedHat team working on this new file system, just the fact that they are trying is interesting.

Edit: and I was at SGI when XFS was pretty new, I know some of the XFS folks as well, Adam Sweeney and Mike Nishimoto.

I was the guy that plugged XFS into NFS over HIPPI, so I have more than a passing knowledge of it:

http://mcvoy.com/lm/bds.pdf

XFS was pretty cool but a lot of the technology that made it fast was XLV, the logical volume manager. XFS just made sure it handed very large, aligned, I/O requests to the volume manager, the volume manager was the layer that split them up and got all the DMA engines going. That's how we did 500MB/sec in the early 1990's on 200mhz MIPS chips, the MIPS chips weren't touching the data, the DMA engines were (the networking stack did page flipping to avoid bcopies).

I know XFS did other stuff for scaling but it most certainly did not do all the safety stuff (and I don't think it did transparent compression, those 200mhz MIPS cpus weren't fast enough to put that in there) that ZFS does.

So I'm wondering how much XFS has evolved from the SGI days. If it hasn't, I don't get why RedHat started there. Be really interested to know the back story.

Post reply on HN