Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Blockdiff: We built our own file format for VM disk snapshots

cognition.ai

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Blockdiff: We built our own file format for VM disk snapshots

#21

Post by hanwenn »

riedel wrote: post #7

I wonder why EC2 is so slow on snapshots. We use CEPH internally and if we wanted we could export the diffs [0] (we use proxmox backup instead). Snapshots always felt blazing fast (I still often forget to do them, need a solution to trigger them from the guest) [0] https://ceph.io/en/news/blog/2013/incremental-snapshots-with...

IIUC, on EC2, the disk (EBS)is one service, and snapshots are in another service (S3). Taking a snapshot involves copying the entire disk to S3, while restoring a snapshot pages blocks in from S3 as the VM accesses the disk.

Re: Blockdiff: We built our own file format for VM disk snapshots

#23

Post by sureglymop »

ori_b wrote: post #16

It's a bit surprising that they dismissed qcow2, because it does exactly what they want. It's also pretty easy to implement in a vm. The file format is a 2 level page table per snapshot, with pointers to blocks. I suspect they didn't really look at it enough. Here's the implementation I did for openbsd; it's around 700 lines, including the gunk to interface with the hypervisor. https://github.com/openbsd/src/blob/mas…

Qcow2 is great for use with vms but I think is actually underrated otherwise.

I use it to back up external disks, usb sticks etc. Because the resulting qcow2 images are sparse and compressed they use less storage which is great for backups.

Re: Blockdiff: We built our own file format for VM disk snapshots

#27

Post by gudmundur »

ori_b wrote: post #16

It's a bit surprising that they dismissed qcow2, because it does exactly what they want. It's also pretty easy to implement in a vm. The file format is a 2 level page table per snapshot, with pointers to blocks. I suspect they didn't really look at it enough. Here's the implementation I did for openbsd; it's around 700 lines, including the gunk to interface with the hypervisor. https://github.com/openbsd/src/blob/mas…

When working with microVM's, which I'm assuming Cognition is doing, you don't have the option of doing qcow2. Raw disks as either block devices or files is all you've got.

Re: Blockdiff: We built our own file format for VM disk snapshots

#28

Post by gudmundur »

pixelbeat__ wrote: post #3

I see you use flags to determine if a file needs syncing. When we used fiemap within GNU cp we required FIEMAP_FLAG_SYNC to get robust operation. (We have since removed the fiemap code from cp, and replaced it with LSEEK_DATA, LSEEK_HOLE)

I've been poking around with both, and FIEMAP has some nice benefits when working with VM disks. For example, if you have the same shared based across multiple VMs, you can use FIEMAP for dirty tracking of blocks, by detecting which extents are no longer shared with the common base. With LSEEK you'll lose that and have to snapshot the entire backing drive.

Re: Blockdiff: We built our own file format for VM disk snapshots

#29

Post by ori_b »

gudmundur wrote: post #27
ori_b wrote: post #16

It's a bit surprising that they dismissed qcow2, because it does exactly what they want. It's also pretty easy to implement in a vm. The file format is a 2 level page table per snapshot, with pointers to blocks. I suspect they didn't really look at it enough. Here's the implementation I did for openbsd; it's around 700 lines, including the gunk to interface with the hypervisor. https://github.com/openbsd/src/blob/mas…

When working with microVM's, which I'm assuming Cognition is doing, you don't have the option of doing qcow2. Raw disks as either block devices or files is all you've got.

That's nonsense. Qcow2 is a file, with a pretty easy to implement format.

Re: Blockdiff: We built our own file format for VM disk snapshots

#30

Post by gudmundur »

ori_b wrote: post #29
gudmundur wrote: post #27

Earlier quoted context omitted.

When working with microVM's, which I'm assuming Cognition is doing, you don't have the option of doing qcow2. Raw disks as either block devices or files is all you've got.

That's nonsense. Qcow2 is a file, with a pretty easy to implement format.

You go give it a try and report back.
Post reply on HN