Live data from Hacker News

Who Needs Git When You Have ZFS?

zef.me

101–110 of 148 posts

Re: Who Needs Git When You Have ZFS?

#101
> Notably missing is support for merging, which ZFS does not have direct support for as far as I'm aware.

So, no replacement. Merge and Branch are the major features of git. The bigger the project the bigger the need for these. I know some projects where people work full time as merge conflict resolvers. Without git that would require a whole team instead of a person.

Re: Who Needs Git When You Have ZFS?

#102
post #73

Earlier quoted context omitted.

Absolutely. It's also good for production. This feature saved my butt once when I upgraded a postgresql 8.3 server to 8.4 on a Sun 4500 not realizing that the new server was compiled with a different timestamp format then the old one was (8 byte integers vs double precision floats). Thanks to ZFS I was able to trivially roll back the update.

That's what docker would help you do too, right? What's the pro/con of zfs compare to docker?

You can use docker on top of ZFS. My employer has something called flocker that can use it:

https://clusterhq.com/flocker/introduction/

The ZFS version is not yet considered a production release due to delays in stabilizing the /dev/zfs API though.

Re: Who Needs Git When You Have ZFS?

#103
post #11

Earlier quoted context omitted.

Yes, if they have ZFS too, it is basically snapshot, send & receive. Or you can clone it and rsync it to them. Do not try to snapshot a running database though. Before you execute the snapshot you should make sure everything is flushed to disk. For example with PostgreSQL you need to create a checkpoint before the snapshot, so you write: SELECT pg_start_backup('prepare_my_snapshot'); AND when the snapshot is done: SE…

> Do not try to snapshot a running database though. Before you execute the snapshot you should make sure everything is flushed to disk. Is a database (or anything else) really supposed to behave that way? Shouldn't every state along the way be possible to resume from? In particular, shouldn't database transactions take care of this?

Yes, it is. One of the fundamental principles of practical RDBMS is durability (the D in https://en.wikipedia.org/wiki/ACID), meaning that when the DB acknowledges a commit to the client, it must be guaranteed that the transaction has been reflected in permanent storage (i.e., on disk).

The fine print: Due to the architecture of file system access in contemporary kernels (esp. file system caches), it's next to impossible for a user-space application to actually guarantee that writes are durable (i.e. synced to disk at a defined point in time), but most RDBMS manage well enough in practice.

Re: Who Needs Git When You Have ZFS?

#104
This blog post was one of the most inspirational pieces that led us to founding Pachyderm (pachyderm.io). We offer git-like semantics for data, including branches, and in a distributed system so it scales to petabytes.

Re: Who Needs Git When You Have ZFS?

#105
post #101

> Notably missing is support for merging, which ZFS does not have direct support for as far as I'm aware. So, no replacement. Merge and Branch are the major features of git. The bigger the project the bigger the need for these. I know some projects where people work full time as merge conflict resolvers. Without git that would require a whole team instead of a person.

"Of course, I'm not seriously suggesting you'd ditch a 'proper' version control system, but it gives a good sense of what's possible at the file system level."

Re: Who Needs Git When You Have ZFS?

#106

"Oracle's (previously Sun's) next-generation file system" That's why you need to stick with Git.

What does Oracle have to do with anything? This is FUD.

For which open-source projects owned by Oracle has Oracle's stewardship been a benefit to users?

Re: Who Needs Git When You Have ZFS?

#107
post #97
post #86

Earlier quoted context omitted.

It's the GPL that's incompatible with the CDDL and not the other way around, which is an important distinction. Other than other GPL variants and BSD and MIT style licenses, the GPL has restrictions which make it pretty much incompatible with everything.

I don't get how the word "incompatible" can be understood as anything other than both license has aspects which makes combining them impossible. But owell, lets see what the license text actually say: Any Covered Software that You distribute or otherwise make available in Executable form must also be made available in Source Code form and that Source Code form must be distributed only under the terms of this License.…

Actually, the CDDL allows the person who wrote the License (Sun, now Oracle) to release a new version of the license that implicitly updates the license for all projects using the old license (It's like the "or any later version" thing with the GPL, but I'm fairly sure it's not optional with CDDL). Which is why people are asking why Oracle doesn't just release CDDL 2.0 that is GPL compatible.

Re: Who Needs Git When You Have ZFS?

#108
post #73

Earlier quoted context omitted.

That's what docker would help you do too, right? What's the pro/con of zfs compare to docker?

Usually data / persistence layer is not stored in Docker images directly. Frequently the data directory is marked in containers as a "volume" to bypass the usual CoW filesystem and write directly to the backing filesystem. So, it's uncommon to snapshot DBs directly in Docker and likely (this is pure speculation) 'zfs snapshot' is more efficient than 'docker commit' (on disk space usage and/or speed of snapshot) for t…

Just to clarify if I'm understanding correctly here: the data itself is often outside Docker for performance and other reasons, but the DB software itself (PostgreSQL / MySQL / whatever) can still easily be within Docker.

Re: Who Needs Git When You Have ZFS?

#109
post #101

> Notably missing is support for merging, which ZFS does not have direct support for as far as I'm aware. So, no replacement. Merge and Branch are the major features of git. The bigger the project the bigger the need for these. I know some projects where people work full time as merge conflict resolvers. Without git that would require a whole team instead of a person.

full time?? wow

Re: Who Needs Git When You Have ZFS?

#110
post #7

ZFS is excellent for database development! Create a snapshot before you try something that mess up your data and instantly restore it! If your development database is a few gigabytes, this will save you a lot of time.

> ZFS is excellent for database development! Create a snapshot before you try something that mess up your data and instantly restore it! If your development database is a few gigabytes, this will save you a lot of time.

The proper way of doing this is to clone the snapshot and then spin up a new container using the clone. That way you do not need to risk downtime on your production code. I believe that Delphix has a ZFS-based solution for doing this.

Post reply on HN