Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Who Needs Git When You Have ZFS?

zef.me

Post reply on HN
Print view View original

111–120 of 148 posts

Re: Who Needs Git When You Have ZFS?

#111

Post by ryao »

belorn wrote: post #97
hhw wrote: post #86

Earlier quoted context omitted.

It's the GPL that's incompatible with the CDDL and not the other way around, which is an important distinction. Other than other GPL variants and BSD and MIT style licenses, the GPL has restrictions which make it pretty much incompatible with everything.

I don't get how the word "incompatible" can be understood as anything other than both license has aspects which makes combining them impossible. But owell, lets see what the license text actually say: Any Covered Software that You distribute or otherwise make available in Executable form must also be made available in Source Code form and that Source Code form must be distributed only under the terms of this License.…

> If we were to make an identical copy of the CDDL license and call it CDDLv2, those two identical twins would be incompatible with each other. Software under CDDLv1 would not be permitted to be combined with software under CDDLv2 and distributed as source code.

There is a CDDL v1.1 that is effectively 's/Sun/Oracle/'. The CDDL has an optional clause saying any later version is allowed and the CDDL only applies at the file level. I have yet to see a problem in mixing CDDL code under v1 and v1.1 with each other.

Re: Who Needs Git When You Have ZFS?

#112

Post by ryao »

cyphar wrote: post #107
belorn wrote: post #97

Earlier quoted context omitted.

I don't get how the word "incompatible" can be understood as anything other than both license has aspects which makes combining them impossible. But owell, lets see what the license text actually say: Any Covered Software that You distribute or otherwise make available in Executable form must also be made available in Source Code form and that Source Code form must be distributed only under the terms of this License.…

Actually, the CDDL allows the person who wrote the License (Sun, now Oracle) to release a new version of the license that implicitly updates the license for all projects using the old license (It's like the "or any later version" thing with the GPL, but I'm fairly sure it's not optional with CDDL). Which is why people are asking why Oracle doesn't just release CDDL 2.0 that is GPL compatible.

New CDDL code outside of Oracle is under CDDLv1 only to prevent them from doing as they please with the license terms.

Re: Who Needs Git When You Have ZFS?

#113

Post by mschuster91 »

ryao wrote: post #110
olavgg wrote: post #7

ZFS is excellent for database development! Create a snapshot before you try something that mess up your data and instantly restore it! If your development database is a few gigabytes, this will save you a lot of time.

> ZFS is excellent for database development! Create a snapshot before you try something that mess up your data and instantly restore it! If your development database is a few gigabytes, this will save you a lot of time. The proper way of doing this is to clone the snapshot and then spin up a new container using the clone. That way you do not need to risk downtime on your production code. I believe that Delphix has a…

> The proper way of doing this is to clone the snapshot

Might not be really feasible. In order to create a consistent snapshot you have to stop the database server (or, if running a multi-master setup, take one master out of replication, but MM setups are rare and even more so is experience with them).

Then, you have to actually do the clone - with a couple-GB-sized database, that's easy and fast but once you hit triple-digits GB sizes, you'll end up with a massive disk performance loss during the copy.

Also, in case your DB upgrade went through, you have to (magically) apply it to the other servers because in the time between clone-begin, clone-end and db-upgrade-end data will have changed on the prod system...

It's easier to make a short maintenance window, do a snapshot, hope for the best and if it blows up, restore service with a single click (by removing the maintenance page).

Re: Who Needs Git When You Have ZFS?

#114

Post by heisnotanalien »

JustSomeNobody wrote: post #30
heisnotanalien wrote: post #17

Earlier quoted context omitted.

Except you'll lose all the data since you created the snapshot... You've been able to do this for years on Linux with LVM anyway.

I don't think you read the comment thoroughly. This is the point. Also, the discussion is centered around ZFS, there's no reason to bring up LVM. Everyone knows there's many ways to skin this cat and it doesn't make anyone appear any smarter because they know of one way.

My point is that it's not really a great way of 'backing' up a database because typically you want the transactions after you've made the change as well.

Re: Who Needs Git When You Have ZFS?

#115

Post by cthalupa »

athenot wrote: post #20

If you look past the attention-grabbing headline, this shows some pretty cool stuff with ZFS, using source versioning as an analogy to explain what would otherwise be quite abstract when it comes to filesystems. The only thing I knew about ZFS was the name, this helped me see some of its impressive features. Of course, I'll take reliability & stability over fancy features any day, but I'm glad there's active developm…

>Of course, I'll take reliability & stability over fancy features any day, but I'm glad there's active development in this area, and look forward to seeing these features mature. They could contribute to some interesting simplifications.

ZFS is in use in production systems all over the world, with a very good reputation for reliability and stability. Nexenta and others have made successful businesses working with large enterprise customers selling ZFS based storage solutions.

Re: Who Needs Git When You Have ZFS?

#117

Post by hodgesrm »

Bromskloss wrote: post #16
olavgg wrote: post #11

Earlier quoted context omitted.

Yes, if they have ZFS too, it is basically snapshot, send & receive. Or you can clone it and rsync it to them. Do not try to snapshot a running database though. Before you execute the snapshot you should make sure everything is flushed to disk. For example with PostgreSQL you need to create a checkpoint before the snapshot, so you write: SELECT pg_start_backup('prepare_my_snapshot'); AND when the snapshot is done: SE…

> Do not try to snapshot a running database though. Before you execute the snapshot you should make sure everything is flushed to disk. Is a database (or anything else) really supposed to behave that way? Shouldn't every state along the way be possible to resume from? In particular, shouldn't database transactions take care of this?

Maybe. RDBMS should be able to recover from any crash consistent copy of storage, where crash consistency means the storage represents a state of the system at run time. FS snapshots provide this property for single volumes.

The problem is that production RDBMS tend to span multiple volumes, e.g., for logs vs. data to pick a simple case. FS snapshots in this case may not be consistent which would cause recovery to fail.

AFAIK this is why even robust RDBMS like MS SQL server use mechanisms like VSS to ensure data are flushed, which guarantees not just crash consistency but application consistency.

Re: Who Needs Git When You Have ZFS?

#118

Post by m45t3r »

urubujj wrote: post #109
erikb wrote: post #101

> Notably missing is support for merging, which ZFS does not have direct support for as far as I'm aware. So, no replacement. Merge and Branch are the major features of git. The bigger the project the bigger the need for these. I know some projects where people work full time as merge conflict resolvers. Without git that would require a whole team instead of a person.

full time?? wow

One unknown person called Linus Torvalds is one of them ;).

Re: Who Needs Git When You Have ZFS?

#119

Post by zenlikethat »

j_s wrote: post #108
zenlikethat wrote: post #77

Earlier quoted context omitted.

Usually data / persistence layer is not stored in Docker images directly. Frequently the data directory is marked in containers as a "volume" to bypass the usual CoW filesystem and write directly to the backing filesystem. So, it's uncommon to snapshot DBs directly in Docker and likely (this is pure speculation) 'zfs snapshot' is more efficient than 'docker commit' (on disk space usage and/or speed of snapshot) for t…

Just to clarify if I'm understanding correctly here: the data itself is often outside Docker for performance and other reasons, but the DB software itself (PostgreSQL / MySQL / whatever) can still easily be within Docker.

Sounds pretty correct, only nitpick is that the data is not "outside of Docker", but rather outside of the Docker image.

Re: Who Needs Git When You Have ZFS?

#120

Post by firethief »

erikb wrote: post #101

> Notably missing is support for merging, which ZFS does not have direct support for as far as I'm aware. So, no replacement. Merge and Branch are the major features of git. The bigger the project the bigger the need for these. I know some projects where people work full time as merge conflict resolvers. Without git that would require a whole team instead of a person.

ZFS could not be a replacement for all of git, but (if you wanted to) you could use it to replace git's storage layer, which is basically independent of operations like merge. I.e. you can git-merge ZFS subvolumes (without any actual git repos) with recursive git-merge-file.
Post reply on HN