Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

curl > /dev/sda: How I made a Linux distro that runs wget | dd

astrid.tech

Post reply on HN
Print view View original

61–70 of 70 posts

Re: curl > /dev/sda: How I made a Linux distro that runs wget | dd

#61

Post by astralbijection »

e12e wrote: post #59

Nice series! Really takes me back to the days of Linux 1.x kernel, Lilo and trying to fit a kernel and initrd on a single floppy disk. So ending up at: > From a 292MB initramfs, we now have a 6.1MB initramfs, smaller than almost every other distro's initramfs and made entirely to run busybox wget dd. Is pretty great achievement today - but way bigger than something that can fit on a floppy.

To be honest, even this has plenty of room to go down. I get the feeling I could have squeezed a couple more MB off if I had actually cut things off of the default Nixpkgs busybox, and possibly also cut a couple of kernel drivers out.

Re: curl > /dev/sda: How I made a Linux distro that runs wget | dd

#62

Post by astralbijection »

klinch wrote: post #39

Sounds cursed. But I'm not judging, given that I use nixos-anywhere[0] on an almost weekly basis. [0] https://github.com/nix-community/nixos-anywhere

Does it make it more cursed that the distro was built off of NixOS Anywhere, and then it theseus shipped NixOS Anywhere out of it?

Re: curl > /dev/sda: How I made a Linux distro that runs wget | dd

#63

Post by motrm »

Mildly pedantic, and of course ignores how wild this whole thing is, but I don't think this bit is correct:

  After waiting for a little while, the program terminated with the following output:
  
  astrid@chungus infra  gzip -vc result/nixos.img | ssh root@myhost.example -- bash -c 'gunzip -vc > /dev/sda'
  root@myhost.example's password:
   77.8% -- replaced with stdout
  
  What happened here?
The 77.8% bit is gunzip -v reporting that it finished decompressing the data to stdout and that the compression ratio was 77.8%... so this invocation may well have succeeded. Assuming, as rwmj points out, nothing else stomped on any of the written blocks.

I do like this idea - with sufficient prep of the system before writing the image, namely stopping as many processes as possible especially those that might do some writing, it's a quick and dirty way to replace a stock OS with a ready-made image. Could perhaps be safer doing it twice, once into a minimal image that does very little beyond network bringup & runs ssh, followed by final OS replacement in a (more) controlled manner.

Re: curl > /dev/sda: How I made a Linux distro that runs wget | dd

#64

Post by rkeene2 »

arboles wrote: post #56
rkeene2 wrote: post #21

Earlier quoted context omitted.

I usually just move all the files to a new directory (/oldroot) and pivot_root -- any open files reference the new paths. Then install into the newly empty root directory of the filesystem, reboot and delete the /oldroot.

Don't you get any errors even if you race immediately to start pivot_root? pivot_root also won't modify all open file descriptors at once. Seems it's not fatal, but have you managed to do this over ssh and not be disconnected?

I don't know what you mean regarding pivot_root affecting file descriptors because they are not modified, they point to new names because the enclosing directory has been moved/renamed. There is a small race between moving items in the root directory as well as after moving all items and before starting pivot_root, but that race doesn't involve file descriptors but opening at the old paths before the new one is established, though lots of things use openat() these days so it doesn't really even occur in most cases then.

Re: curl > /dev/sda: How I made a Linux distro that runs wget | dd

#66

Post by astralbijection »

megous wrote: post #47

Instead of applying some sense to the problem, and using a solution that actually allows you to kill all running processes of the original distro at runtime, incl. getting rid of the original init process, to be able to pivot_root somewhere else amd umount the original system's filesystems and free the block device for re-installation, this ridiculous approach gets promoted to a front page, lol.

I did learn about systemctl switch-root after finishing my kexec solution, but I think if you're at the point of reimaging a server over the network, you probably don't care enough to shut it down correctly :)

Re: curl > /dev/sda: How I made a Linux distro that runs wget | dd

#67

Post by justsomehnguy »

e12e wrote: post #59

Nice series! Really takes me back to the days of Linux 1.x kernel, Lilo and trying to fit a kernel and initrd on a single floppy disk. So ending up at: > From a 292MB initramfs, we now have a 6.1MB initramfs, smaller than almost every other distro's initramfs and made entirely to run busybox wget dd. Is pretty great achievement today - but way bigger than something that can fit on a floppy.

Modern "floppy" is just any USB thumbdrive you found lying around. Just giggles checked my 'to go' retailer - there are whopping 3 variants of 4GB no-name drives for $6. For $12 you would get a Kingston one with 64GB.

"Floppy" image size is not important now nor for the last 15 years.

Re: curl > /dev/sda: How I made a Linux distro that runs wget | dd

#68

Post by tiberious726 »

ahartmetz wrote: post #37
tux3 wrote: post #30

Earlier quoted context omitted.

I used to dual-boot windows, but I was too lazy to actually reboot, so naturally I had Virtualbox just boot the physical Windows partition while Linux was running. Which is totally fine! It's not a real dual boot if you don't boot both partitions at the same time. As long as you don't install guest VBox drivers, those would make it hang when it boots as the host on physical hardware, since there's no longer someone a…

I think Windows refused to do that at some point? So I booted the physical Linux partition from Windows if I needed both at the same time. That's on a laptop that otherwise almost always ran Linux.

It whines about licensing, but I switch between booting my windows installation bare metal and as a VM all the time

Re: curl > /dev/sda: How I made a Linux distro that runs wget | dd

#69

Post by megous »

astralbijection wrote: post #66
megous wrote: post #47

Instead of applying some sense to the problem, and using a solution that actually allows you to kill all running processes of the original distro at runtime, incl. getting rid of the original init process, to be able to pivot_root somewhere else amd umount the original system's filesystems and free the block device for re-installation, this ridiculous approach gets promoted to a front page, lol.

I did learn about systemctl switch-root after finishing my kexec solution, but I think if you're at the point of reimaging a server over the network, you probably don't care enough to shut it down correctly :)

Actually you do, bacause you can't re-image safely while kernel is also writing to your target block device via fs driver. And if original server has nything more complex going on, like md/lvm you're in even more trouble.

Re: curl > /dev/sda: How I made a Linux distro that runs wget | dd

#70

Post by arboles »

rkeene2 wrote: post #64
arboles wrote: post #56

Earlier quoted context omitted.

Don't you get any errors even if you race immediately to start pivot_root? pivot_root also won't modify all open file descriptors at once. Seems it's not fatal, but have you managed to do this over ssh and not be disconnected?

I don't know what you mean regarding pivot_root affecting file descriptors because they are not modified, they point to new names because the enclosing directory has been moved/renamed. There is a small race between moving items in the root directory as well as after moving all items and before starting pivot_root, but that race doesn't involve file descriptors but opening at the old paths before the new one is estab…

I didn't know file descriptors worked like that admittedly. Also instead of moving the files couldn't we copy them to the new root and pivot_root then?
Post reply on HN