Blockdiff: We built our own file format for VM disk snapshots
1–10 of 30 posts
Re: Blockdiff: We built our own file format for VM disk snapshots
#2Can this be used in a public cloud provider , to speed up VM provisioning in CI/CD pipelines? Im looking for ways to speed up app provisioning for e2e tests.
Re: Blockdiff: We built our own file format for VM disk snapshots
#3I see you use flags to determine if a file needs syncing. When we used fiemap within GNU cp we required FIEMAP_FLAG_SYNC to get robust operation.
(We have since removed the fiemap code from cp, and replaced it with LSEEK_DATA, LSEEK_HOLE)
Re: Blockdiff: We built our own file format for VM disk snapshots
#4Thanks for writing the blog post; it was a fascinating read!
I was curious about a couple of things:
* Have you considered future extensions where you can start the VM before you completed the FS copy?
* You picked XFS over ZFS and BTRFS. Any reason why XFS in particular?
* You casually mention that you wrote 'otterlink', your own hypervisor. Isn't that by itself a complicated effort worthy of a blog post? Or is it just mixing and matching existing libraries from the Rust ecosystem?
Re: Blockdiff: We built our own file format for VM disk snapshots
#5Have you considered https://github.com/containerd/overlaybd? It seems to offer very similar features to blockdiff.
Re: Blockdiff: We built our own file format for VM disk snapshots
#6Thanks for writing the blog post; it was a fascinating read! I was curious about a couple of things: * Have you considered future extensions where you can start the VM before you completed the FS copy? * You picked XFS over ZFS and BTRFS. Any reason why XFS in particular? * You casually mention that you wrote 'otterlink', your own hypervisor. Isn't that by itself a complicated effort worthy of a blog post? Or is it j…
Not the author, but:
> Any reason why XFS in particular?
XFS is still the default filesystem of choice for many enterprise systems. For instance, Red Hat states in their manual [1]:
In general, use XFS unless you have a specific use case for ext4.
[1] https://docs.redhat.com/en/documentation/red_hat_enterprise_...
Re: Blockdiff: We built our own file format for VM disk snapshots
#7I wonder why EC2 is so slow on snapshots. We use CEPH internally and if we wanted we could export the diffs [0] (we use proxmox backup instead). Snapshots always felt blazing fast (I still often forget to do them, need a solution to trigger them from the guest)
[0] https://ceph.io/en/news/blog/2013/incremental-snapshots-with...
Re: Blockdiff: We built our own file format for VM disk snapshots
#8The blog's title can be misleading here, "we" in this context refers to the Cognition team. I don't work at Cognition, just thought this was interesting.
Re: Blockdiff: We built our own file format for VM disk snapshots
#9Thanks for writing the blog post; it was a fascinating read! I was curious about a couple of things: * Have you considered future extensions where you can start the VM before you completed the FS copy? * You picked XFS over ZFS and BTRFS. Any reason why XFS in particular? * You casually mention that you wrote 'otterlink', your own hypervisor. Isn't that by itself a complicated effort worthy of a blog post? Or is it j…
Also: did you consider using LVM snapshots?
Re: Blockdiff: We built our own file format for VM disk snapshots
#10I don’t know how to express to you how stupid, inadequate, and envious I feel of this level of competence.
For me this article has the density of slaps of a plutonium ingot.
It’s moving to read (and “maybe” understand, given how well it’s written).
Wow, maximum respect, truly.