Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Wipe and reinstall a running Linux system via SSH (2017)

github.com

Post reply on HN
Print view View original

51–60 of 81 posts

Re: Wipe and reinstall a running Linux system via SSH (2017)

#51

Post by vdloo »

We upgraded a whole fleet of AWS / Digitalocean instances without floating IPs from Ubuntu Precise to Xenial based on this method back in the summer of 2017. While obviously not having to do crazy stuff like this would be better, it's nice to know that it is possible if you really need it.

https://www.reddit.com/r/programming/comments/6o7i8p/dont_ru...

Recently we ran into another use-case for this in production actually, we needed to wipe a lot of servers in our datacenter remotely and we figured one of the options would be to install some OS in memory with the relevant wiping tools, pivot_root to that, unmount all disks and then perform the wipe. In the end we went a different route and opted for a custom PXE-boot image instead that the servers would boot into that scripted the whole thing.

Re: Wipe and reinstall a running Linux system via SSH (2017)

#52

Post by vbezhenar »

filmor wrote: post #17
vbezhenar wrote: post #12

Earlier quoted context omitted.

It's not trivial and also it's likely to be very poorly tested scenario. I would recreate FS even if it supports resize.

For a lot of filesystems, growing is indeed trivial and even shrinking is explicitly supported. Why would this be less tested than any other feature?

Because users put their systems under various I/O workloads every day, but very few people resize their file systems and they do it very rarely.

Re: Wipe and reinstall a running Linux system via SSH (2017)

#53

Post by mirimir »

zamadatix wrote: post #50
mirimir wrote: post #45

> This script does not have any provisions for exiting out of the new environment back into something sane. You will have to reboot when you're done. If you get anything wrong, your machine won't boot. Tough luck. So then "without rebooting" in the title is inaccurate, no? I haven't studied TFA carefully. But reinstalling running Linux systems via SSH is pretty routine, no? Using debian-installer with network-console…

Wipe and reinstall, not reinstall and load/run. I think the title is fair.

I guess. But after reinstalling with LUKS, the box is pretty thoroughly wiped.

Re: Wipe and reinstall a running Linux system via SSH (2017)

#54

Post by pbhjpbhj »

kilburn wrote: post #43
marcosdumay wrote: post #35

Earlier quoted context omitted.

The "good" Linux answer would be to do the equivalent to your script on parted. I have no idea why an explanation of how to remotely install Linux on a machine while running Linux become the accepted answer. But about Windows, last year I had to resize the partitions on my work desktop, there are enough restrictions around changing things on the disk that Windows is installed that it became a week long research task…

> I have no idea why an explanation of how to remotely install Linux [mess with partitions] on a machine while running Linux The question was explicitly about shrinking the root filesystem without booting a livecd or any other OS. Gparted cannot do that directly because the root filesystem cannot be unmounted, and the appropriate answer for nearly everyone else (boot from a livecd and use gparted) doesn't apply becau…

The SO answer creates a root environment and uses `pivot_root`, seems you could "just" duplicate the root filesystem, pivot to that full copy, then unmount the original root and do what you wanted with it?

Re: Wipe and reinstall a running Linux system via SSH (2017)

#55

Post by cliffy »

mirimir wrote: post #45

> This script does not have any provisions for exiting out of the new environment back into something sane. You will have to reboot when you're done. If you get anything wrong, your machine won't boot. Tough luck. So then "without rebooting" in the title is inaccurate, no? I haven't studied TFA carefully. But reinstalling running Linux systems via SSH is pretty routine, no? Using debian-installer with network-console…

Maybe kexec-ing into the new kernel would be feasible?

I investigated this many years ago to see if it was, but I found scant information on compatibility requirements when using kexec to hand over execution to an arbitrary kernel.

The problem seems really hard though. One issue that stands out to me is that even if you properly shutdown the old kernel, will all system devices be in a 'good enough' state to be reinitialized properly by the new one? Or do some devices require a reboot for some reason?

Re: Wipe and reinstall a running Linux system via SSH (2017)

#56

Post by arminiusreturns »

mirimir wrote: post #45

> This script does not have any provisions for exiting out of the new environment back into something sane. You will have to reboot when you're done. If you get anything wrong, your machine won't boot. Tough luck. So then "without rebooting" in the title is inaccurate, no? I haven't studied TFA carefully. But reinstalling running Linux systems via SSH is pretty routine, no? Using debian-installer with network-console…

I do this method also (for personal projects I host in vultr/do), I call it "initram shimming"

Re: Wipe and reinstall a running Linux system via SSH (2017)

#57

Post by mirimir »

cliffy wrote: post #55
mirimir wrote: post #45

> This script does not have any provisions for exiting out of the new environment back into something sane. You will have to reboot when you're done. If you get anything wrong, your machine won't boot. Tough luck. So then "without rebooting" in the title is inaccurate, no? I haven't studied TFA carefully. But reinstalling running Linux systems via SSH is pretty routine, no? Using debian-installer with network-console…

Maybe kexec-ing into the new kernel would be feasible? I investigated this many years ago to see if it was, but I found scant information on compatibility requirements when using kexec to hand over execution to an arbitrary kernel. The problem seems really hard though. One issue that stands out to me is that even if you properly shutdown the old kernel, will all system devices be in a 'good enough' state to be reinit…

I have no clue.

When I've built custom kernels, I'm pretty sure that the new kernel wasn't active until I rebooted. But rebooting after even dist-upgrade has just become automatic.

For unattended upgrades, you can disable automatic reboot. But then, I think there's risk that some upgrades won't take effect.

Re: Wipe and reinstall a running Linux system via SSH (2017)

#58

Post by kilburn »

pbhjpbhj wrote: post #54
kilburn wrote: post #43

Earlier quoted context omitted.

> I have no idea why an explanation of how to remotely install Linux [mess with partitions] on a machine while running Linux The question was explicitly about shrinking the root filesystem without booting a livecd or any other OS. Gparted cannot do that directly because the root filesystem cannot be unmounted, and the appropriate answer for nearly everyone else (boot from a livecd and use gparted) doesn't apply becau…

The SO answer creates a root environment and uses `pivot_root`, seems you could "just" duplicate the root filesystem, pivot to that full copy, then unmount the original root and do what you wanted with it?

You can remount the root filesystem elsewhere, but not unmount it while it is still the rootfs. Your suggestion would work but it involves restarting nearly everything and having another partition with enough space to copy everything into. It can be done but is pretty hard in general, so gparted just doesn't support it. If you really want/need to do it, you can do so. Otherwise just use a livecd and make your life easier :)

Re: Wipe and reinstall a running Linux system via SSH (2017)

#59

Post by pbhjpbhj »

kilburn wrote: post #58
pbhjpbhj wrote: post #54

Earlier quoted context omitted.

The SO answer creates a root environment and uses `pivot_root`, seems you could "just" duplicate the root filesystem, pivot to that full copy, then unmount the original root and do what you wanted with it?

You can remount the root filesystem elsewhere, but not unmount it while it is still the rootfs. Your suggestion would work but it involves restarting nearly everything and having another partition with enough space to copy everything into. It can be done but is pretty hard in general, so gparted just doesn't support it. If you really want/need to do it, you can do so. Otherwise just use a livecd and make your life ea…

I assumed that the point was it's a remote situation and so mounting a liveUSB would be costly.

Re: Wipe and reinstall a running Linux system via SSH (2017)

#60

Post by andrewflnr »

xorcist wrote: post #30
mehrdadn wrote: post #3

Earlier quoted context omitted.

I love how the first comment praises it for being "straightforward". For reference, this is how you shrink a file system on Windows: diskpart select volume C: shrink desired=4096 I guess it's nice that the Linux version lets you move the partition as well (or "shrink from behind", so to speak), but damn, "straightforward" is not a word I would have used to describe it.

And this is how to shrink a mounted filesystem in Linux when you use btrfs: btrfs filesystem resize 4g / It's just not supported for ext4. You can grow an ext4 file system, even when it is your currently mounted root, just not shrink it. While it's great that NTFS does (now) support it few other file systems do because it's a relatively uncommon and quite risky. I certainly wouldn't shrink mounted filesystems even on…

You can totally shrink an ext4 filesystem.
Post reply on HN