Who Needs Git When You Have ZFS? • Hacker News Board

Re: Who Needs Git When You Have ZFS?

#71

Post by espadrine » Wed, May 25, 2016, 2:26 PM UTC

olavgg wrote: post #7

ZFS is excellent for database development! Create a snapshot before you try something that mess up your data and instantly restore it! If your development database is a few gigabytes, this will save you a lot of time.

I have read that COW filesystems tend to perform terribly on HDD with database workloads because overwrites mid-file are common and increase external fragmentation.

Is ZFS any better than Btrfs in this regard?

Re: Who Needs Git When You Have ZFS?

#72

Post by eeeeeeeeeeeee » Wed, May 25, 2016, 2:27 PM UTC

raverbashing wrote: post #5

> Who needs Git? > Notably missing is support for merging So, it's almost like a car except it's missing its wheels No, it's not like Git

Maybe read the article before adding a snarky comment?

Re: Who Needs Git When You Have ZFS?

#73

Post by tkinom » Wed, May 25, 2016, 2:32 PM UTC

savedmybutt wrote: post #44

olavgg wrote: post #7

ZFS is excellent for database development! Create a snapshot before you try something that mess up your data and instantly restore it! If your development database is a few gigabytes, this will save you a lot of time.

Absolutely. It's also good for production. This feature saved my butt once when I upgraded a postgresql 8.3 server to 8.4 on a Sun 4500 not realizing that the new server was compiled with a different timestamp format then the old one was (8 byte integers vs double precision floats). Thanks to ZFS I was able to trivially roll back the update.

That's what docker would help you do too, right?

What's the pro/con of zfs compare to docker?

Re: Who Needs Git When You Have ZFS?

#74

Post by espadrine » Wed, May 25, 2016, 2:34 PM UTC

ryao wrote: post #64

espadrine wrote: post #58

Earlier quoted context omitted.

Exactly. The legal situation of Google's use of Java APIs was always a tiny bit murky, but it was super-clear-cut compared to the use of ZFS-on-Linux.

Oracle sued over Google reimplementing code that was under the GPL (OpenJDK) under the Apache 2.0 license (Apache Harmony). Google recently switched to the GPL code because of it. In the case of ZoL, the code is derived from the original code and is under the original license. The idea that ZoL is somehow more at risk is pure FUD.

My understanding is that the risk is mostly the other way around: it infringes Linux' GPL license (see https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/). Would Oracle have grounds to sue as a Linux copyright holder? They have successfully made API copyrightable, so they clearly have competent lawyers.

Re: Who Needs Git When You Have ZFS?

#75

Post by lazyant » Wed, May 25, 2016, 2:41 PM UTC

I've been reading that ZFS was "almost ready for use" in Linux for years, how stable is it for production now, anybody here using it, any good/bad comments or tips? (using CentOS 7 at the moment)

Re: Who Needs Git When You Have ZFS?

#76

Post by 0xCMP » Wed, May 25, 2016, 2:43 PM UTC

tkinom wrote: post #73

savedmybutt wrote: post #44

Earlier quoted context omitted.

Absolutely. It's also good for production. This feature saved my butt once when I upgraded a postgresql 8.3 server to 8.4 on a Sun 4500 not realizing that the new server was compiled with a different timestamp format then the old one was (8 byte integers vs double precision floats). Thanks to ZFS I was able to trivially roll back the update.

That's what docker would help you do too, right? What's the pro/con of zfs compare to docker?

ZFS is what the filesystem your volume would be using. You should not store ("commit" really) the database data in a Docker image.

Re: Who Needs Git When You Have ZFS?

#77

Post by zenlikethat » Wed, May 25, 2016, 2:44 PM UTC

tkinom wrote: post #73

savedmybutt wrote: post #44

Earlier quoted context omitted.

Absolutely. It's also good for production. This feature saved my butt once when I upgraded a postgresql 8.3 server to 8.4 on a Sun 4500 not realizing that the new server was compiled with a different timestamp format then the old one was (8 byte integers vs double precision floats). Thanks to ZFS I was able to trivially roll back the update.

That's what docker would help you do too, right? What's the pro/con of zfs compare to docker?

Usually data / persistence layer is not stored in Docker images directly. Frequently the data directory is marked in containers as a "volume" to bypass the usual CoW filesystem and write directly to the backing filesystem. So, it's uncommon to snapshot DBs directly in Docker and likely (this is pure speculation) 'zfs snapshot' is more efficient than 'docker commit' (on disk space usage and/or speed of snapshot) for this workload due to different use cases.

I'd be really curious to see some actual numbers on this though.

Re: Who Needs Git When You Have ZFS?

#78

Post by johannes1234321 » Wed, May 25, 2016, 2:46 PM UTC

espadrine wrote: post #71

olavgg wrote: post #7

ZFS is excellent for database development! Create a snapshot before you try something that mess up your data and instantly restore it! If your development database is a few gigabytes, this will save you a lot of time.

I have read that COW filesystems tend to perform terribly on HDD with database workloads because overwrites mid-file are common and increase external fragmentation. Is ZFS any better than Btrfs in this regard?

The answer to that is: use an SSD. With SSD random seek vs sequential access doesn't matter. Second part of the answer is that ZFS provides features for cache devices. So you get a fast disk as cache and a set of larger disks for as actual storage. All writes at first go to the cache device to be safe and then are written to actual storage, thanks to cow this can be done in larger sequential sequences. (Unless the disks become too full, ZFS has issues when disks are ~80% full, then it takes too much time to find free space) What's left are longer "sequential" read operations. There ZFS tries to predict the read, this can be successful, but might also happen that the seeks take too much time for highest performance.

Re: Who Needs Git When You Have ZFS?

#79

Post by ElijahLynn » Wed, May 25, 2016, 2:49 PM UTC

lazyant wrote: post #75

I've been reading that ZFS was "almost ready for use" in Linux for years, how stable is it for production now, anybody here using it, any good/bad comments or tips? (using CentOS 7 at the moment)

To give you an idea, it just shipped baked into Ubuntu 16.04, the latest Long Term Support (LTS) version.

https://insights.ubuntu.com/2016/02/16/zfs-is-the-fs-for-con...

Re: Who Needs Git When You Have ZFS?

#80

Post by mkhpalm » Wed, May 25, 2016, 2:59 PM UTC

olavgg wrote: post #7

ZFS is excellent for database development! Create a snapshot before you try something that mess up your data and instantly restore it! If your development database is a few gigabytes, this will save you a lot of time.

True, but you can/could easily do that with LVM snapshots and merges too. We snapshot the databases (copy on write) which is instant, then if we want to roll it back we lvm merge and remount. Everything is pretty instant and it doesn't matter what filesystem you want to use.