Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Timeshift: System Restore Tool for Linux

github.com

Post reply on HN
Print view View original

141–150 of 180 posts

Re: Timeshift: System Restore Tool for Linux

#141

Post by magicalhippo »

abbbi wrote: post #119
magicalhippo wrote: post #91

Earlier quoted context omitted.

Any backup software that utilizes LVM in this way? Ie automatically creates a snapshot and sends the incremental changes since previous snapshot to a backup destination like a NAS or S3 blob storage.

wyng backup does this. It uses the device mappers thin_dump tools to allow for incremental backups between snapshots, too: https://github.com/tasket/wyng-backup edit: requires lvm thin provisioned volumes There is also thin-send-recv which basically does the same as zfs send/recv just with lvm: https://github.com/LINBIT/thin-send-recv it uses the same functions of the device mapper to allow incremental sync of lvm th…

Thanks for the pointers, looks very relevant.

It's just such a low-effort peace of mind. Just a few clicks and I know that regardless what happens to my disk or my system, I can be up and running in very little time with very little effort.

On Linux it's always a bit more work, but backups and restore is one of those things I prefer is not too complicated, as stress level is usually high enough when you need to do restore to worry about forgetting some incantation steps.

Re: Timeshift: System Restore Tool for Linux

#142

Post by aeadio »

sieve wrote: post #126
rabf wrote: post #114

Earlier quoted context omitted.

Best option to date: https://github.com/zbm-dev/zfsbootmenu A shame most distro's installers don't support it natively, but an encrypted rootfs on ZFS is great once you get it setup.

Yeah. I am somewhat wary of trying this, mucking something up and wasting a lot of time wrestling with it. Will probably play around with it in a vm and use it during the next ssd upgrade. Would have been so much better if the distros showed more interest in ZFS

In principle there's no reason you can't install this next to GRUB in case you're wary. If you're not using ZFS native encryption, and make sure not to enable some newer zpool features, GRUB booting should work for ZFS-on-root.

That said, I've been using the tool for a while now and it's been really rock solid. And once you have it installed and working, you don't really have to touch it again, until some hypothetical time when a new backward-incompatible zpool feature gets added that you want to use, and you need a newer ZFSBootMenu build to support it.

Because it's just an upstream Linux kernel with the OpenZFS kmod, and a small dracut module to import the pool and display a TUI menu, it's mechanically very simple, and relying on core ZFS support in the Linux kernel module and userspace that's already pretty battle tested.

After seeing people in IRC try to diagnose recent GRUB issues with very vanilla setups (like ext4 on LVM), I'm becoming more and more convinced that the general approach used by ZFSBootMenu is the way to go for modern EFI booting. Why maintain a completely separate implementation of all the filesystems, volume managers, disk encryption technologies, when a high quality reference implementation already exists in the kernel? The kernel knows how to boot itself, unlock and mount pretty much any combination of filesystem and volume manager, and then kexec the kernel/initrd inside.

The upsides to ZFSBootMenu, OTOH,

    * Supports all ZFS features from the most recent OpenZFS versions, since it uses the OpenZFS kmod 
    * Select boot environment (and change the default boot environment) right from the boot loader menu
    * Select specific kernels within each boot environment (and change the default kernel)  
    * Edit kernel command line temporarily  
    * Roll back boot environments to a previous snapshot  
    * Rewind to a pool checkpoint  
    * Create, destroy, promote and orphan boot environments  
    * Diff boot environments to some previous snapshot to see all file changes
    * View pool health / status  
    * Jump into a chroot of a boot environment  
    * Get a recovery shell with a full suite of tools available including zfs and zpool, in addition to many helper scripts for managing your pool/datasets and getting things back into a working state before either relaunching the boot menu, or just directly booting into the selected dataset/kernel/initrd pair.
    * Even supports user mode SecureBoot signing -- you just need to pass the embedded dracut config the right parameters to produce a unified image, and sign it with your key of choice. No need to mess around with shim and separate kernel signing.

Re: Timeshift: System Restore Tool for Linux

#143

Post by magicalhippo »

_flux wrote: post #104
magicalhippo wrote: post #91

Earlier quoted context omitted.

Any backup software that utilizes LVM in this way? Ie automatically creates a snapshot and sends the incremental changes since previous snapshot to a backup destination like a NAS or S3 blob storage.

I think block-level snapshots would be very difficult to use this way. I just make a full dedupped backups from LVM snapshots with kopia, but I've set that up only on one system, on others I just use kopia as-is. It takes some time, but that's fine for me. Previous backup of 25 GB an hour ago took 20 minutes. I suppose if it only walked files it knew were changed it would be a lot faster.

Thanks, sounds interesting. So you create a snapshot, then let kopia process that snapshot rather than the live filesystem, and then remove the snapshot?

> I suppose if it only walked files it knew were changed it would be a lot faster.

Right, for me I'd want to set it up to do the full disk, so could be millions of files and hundreds of GB. But this trick should work with other backups software, so perhaps it's a viable option.

Re: Timeshift: System Restore Tool for Linux

#144

Post by magicalhippo »

lmz wrote: post #95
magicalhippo wrote: post #91

Earlier quoted context omitted.

Any backup software that utilizes LVM in this way? Ie automatically creates a snapshot and sends the incremental changes since previous snapshot to a backup destination like a NAS or S3 blob storage.

I don't think the diffs are usable that way. They're actually more like an "undo log" in that the snapshot space is taken by "old blocks" when the actual volume is taking writes. It's useful for the same reasons as volume shadow copy: a consistent snapshot of the block device. (Also this can be very bad for write performance as any writes are doubled - to snapshot and to to the real device)

Yeah ok, that makes sense. Write performance is a concern, but usually the backups run when there's little activity.

Re: Timeshift: System Restore Tool for Linux

#145

Post by abbbi »

magicalhippo wrote: post #141
abbbi wrote: post #119

Earlier quoted context omitted.

wyng backup does this. It uses the device mappers thin_dump tools to allow for incremental backups between snapshots, too: https://github.com/tasket/wyng-backup edit: requires lvm thin provisioned volumes There is also thin-send-recv which basically does the same as zfs send/recv just with lvm: https://github.com/LINBIT/thin-send-recv it uses the same functions of the device mapper to allow incremental sync of lvm th…

Thanks for the pointers, looks very relevant. It's just such a low-effort peace of mind. Just a few clicks and I know that regardless what happens to my disk or my system, I can be up and running in very little time with very little effort. On Linux it's always a bit more work, but backups and restore is one of those things I prefer is not too complicated, as stress level is usually high enough when you need to do re…

it depends. Doing a complete disaster recovery of a windows system IMHO can be a real struggle. Especially if you have to restore a system to different hardware, which the system state backup that microsoft offers does not support afaik.

Backing up a linux system in combination with REAR:

https://github.com/rear/rear

and a backup utility of your choice for the regular backup has never failed me so far. I used it to restore linux systems to complete different hardware without any troubles.

Re: Timeshift: System Restore Tool for Linux

#146

Post by _flux »

magicalhippo wrote: post #143
_flux wrote: post #104

Earlier quoted context omitted.

I think block-level snapshots would be very difficult to use this way. I just make a full dedupped backups from LVM snapshots with kopia, but I've set that up only on one system, on others I just use kopia as-is. It takes some time, but that's fine for me. Previous backup of 25 GB an hour ago took 20 minutes. I suppose if it only walked files it knew were changed it would be a lot faster.

Thanks, sounds interesting. So you create a snapshot, then let kopia process that snapshot rather than the live filesystem, and then remove the snapshot? > I suppose if it only walked files it knew were changed it would be a lot faster. Right, for me I'd want to set it up to do the full disk, so could be millions of files and hundreds of GB. But this trick should work with other backups software, so perhaps it's a vi…

Exactly so.

Here's the script, should it be of benefit to someone, even if it of course needs to be modified:

    #!/bin/sh
    success=false
    teardown() {
      umount /mnt/backup/var/lib/docker || true
      umount /mnt/backup/root/.cache || true
      umount /mnt/backup/ || true
      for lv in root docker-data; do
        lvremove --yes /dev/hass-vg/$lv-snapshot || true
      done
    
      if [ "$1" != "no-exit" ]; then
        $success
        exit $?
      fi
    }
    
    set -x
    set -e
    teardown no-exit
    trap teardown EXIT
    for lv in root docker-data; do
      lvcreate --snapshot -L 1G -n $lv-snapshot /dev/hass-vg/$lv
    done
    
    mount /dev/hass-vg/root-snapshot /mnt/backup
    mount /dev/hass-vg/docker-data-snapshot /mnt/backup/var/lib/docker
    mount /root/.cache /mnt/backup/root/.cache -o bind
    
    chroot /mnt/backup kopia --config-file="/root/.config/kopia/repository.config" --log-dir="/root/.cache/kopia" snap create / /var/lib/docker
    kopia --config-file="/root/.config/kopia/repository.config" --log-dir="/root/.cache/kopia" snap create /boot /boot/efi
    success=true

Re: Timeshift: System Restore Tool for Linux

#147

Post by setopt »

pixelmonkey wrote: post #139
setopt wrote: post #132

Earlier quoted context omitted.

I’ve also found Arq to be brilliant on MacOS. It’s especially nice on laptops, where you can e.g. set it to pause on battery and during working hours. Also, APFS snapshots is a nice thing given how many Mac apps use SQLite databases under the hood (Photos, Notes, Mail, etc.). On Linux, the system I liked best was rsnapshot: I love its brutal simplicity (cron + rsync + hardlinks), and how easy it is to browse previous…

rsnapshot was definitely my favorite Linux option before restic. I find that restic gives me the benefits of chunk-based deduplication and encryption, but via `restic find` and `restic mount` I can also get many of the benefits of rsnapshot's simplicity. If you use `restic mount` against a local repo on a USB3 SSD, the FUSE filesystem is actually pretty fast.

Thanks for the info, I’ll have a closer look at Restic then. Borg also has a FUSE interface, but last time I tried it I found it abysmally slow – much slower than just restoring a folder to disk and then grepping through it. I used a Raspberry Pi as my backup server though, so the FUSE was perhaps CPU bound on my system.

Re: Timeshift: System Restore Tool for Linux

#149

Post by pixelmonkey »

setopt wrote: post #147
pixelmonkey wrote: post #139

Earlier quoted context omitted.

rsnapshot was definitely my favorite Linux option before restic. I find that restic gives me the benefits of chunk-based deduplication and encryption, but via `restic find` and `restic mount` I can also get many of the benefits of rsnapshot's simplicity. If you use `restic mount` against a local repo on a USB3 SSD, the FUSE filesystem is actually pretty fast.

Thanks for the info, I’ll have a closer look at Restic then. Borg also has a FUSE interface, but last time I tried it I found it abysmally slow – much slower than just restoring a folder to disk and then grepping through it. I used a Raspberry Pi as my backup server though, so the FUSE was perhaps CPU bound on my system.

Yea, I don't want to oversell it. The restic FUSE mount isn't anywhere near "native" performance. But, it's fast enough that if you can narrow your search to a directory, and if you're using a local restic repo, using grep and similar tools is do-able. To me, using `restic mount` over a USB3 SSD repo makes the mount folder feel sorta like a USB2 filesystem rather than a USB3 one.

Re: Timeshift: System Restore Tool for Linux

#150

Post by jlokier »

hashworks wrote: post #76
ratorx wrote: post #48

Earlier quoted context omitted.

One problem with file based backups is that they are not atomic across the filesystem. If you ever back up a database (or really any application that expects atomicity while it’s running), then you might corrupt the database and lose data. This might not seem like a big problem, but can affect e.g. SQLite, which is quite popular as a file format. Then again, the likelihood that the backup will be inconsistent is fair…

While I do that, is that really the case? I can imagine database snapshots are consistent most of the time, but it can't be guaranteed, right? In the end it's like a server crash, the database suddenly stops.

That works if the backup uses a snapshot of the filesystem or a point in time. Then the backup state is equivalent to what you'd get if the server suddenly lost power, which all good ACID databases handle.

The GP is talking about when the backup software reads database files gradually from the live filesystem at the same time as the database is writing the same files. This can result in an inconsistent "sliced" state in the backup, which is different from anything you get if the database crashes or the system crashes or loses power.

The effect is a bit like when "fsync" and write barriers are not used before a server crash, and an inconsistent mix of things end up in the file. Even databases that claim to be append-only and resistant to this form of corruption usually have time windows where they cannot maintain that guarantee, e.g. when recycling old log space if the backup process is too slow.

Post reply on HN