Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloning a Laptop over NVMe TCP

copyninja.in

Post reply on HN
Print view View original

21–30 of 181 posts

Re: Cloning a Laptop over NVMe TCP

#21

Post by tripflag »

mrb wrote: post #5

In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…

This use of dd may cause corruption! You need iflag=fullblock to ensure it doesn't truncate any blocks, and (at the risk of cargo-culting) conv=sync doesn't hurt as well. I prefer to just nc -l -p 1234 > /dev/nvme0nX.

Re: Cloning a Laptop over NVMe TCP

#22

Post by jesprenj »

> and I'm not so familiar with resizing LUKS.

LUKS2 does not care about the size of the disk. If it's JSON header is present, it will by default treat the entire underlying block device as the LUKS encrypted volume/partition (sans the header), unless specified otherwise on the commandline.

Re: Cloning a Laptop over NVMe TCP

#23

Post by dmos62 »

I recently had to copy around 200gb of files over wifi. I used rsync to make sure a connection failure doesn't mean I have to start over and so that nothing is lost, but it took at least 6 hours. I wonder what could I have done better.

Btw, what kinds of guarantees do you get with the dd method? Do you have to compare md5s of the resulting block level devices after?

Re: Cloning a Laptop over NVMe TCP

#24

Post by bayindirh »

mrb wrote: post #5

In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…

As a sysadmin, I'd rather use NVMe TCP or Clonezilla to do a slow write rather than trying to go 5% faster with more moving parts and chance to corrupt my drive in the process.

Plus, a it'd be well deserved coffee break.

Considering I'd be going at GigE speeds at best, I'd add "oflag=direct" to bypass caching on the target. A bog standard NVMe can write >300MBps unhindered, so trying to cache is moot.

Lastly, parted can do partition resizing, but given the user is not a power user to begin with, it's just me nitpicking. Nice post otherwise.

Re: Cloning a Laptop over NVMe TCP

#25

Post by justsomehnguy »

jauntywundrkind wrote: post #14
wjnc wrote: post #6

So this just bit for bit dumps a NVMe device to another location. That’s clear. So all encryption just is transferred and not touched. But doesn’t the next machine go into panic when you boot? There are probably many changes in the underlying machine? (Okay, now I read the other post. The author really knows the way. This is at least intermediate Linux.)

A Linux install is often remarkably hardware agnostic. Windows would panic, certainly (because so much drivers & other state is persisted & expected), but the Linux kernel when it boots kind of figures out afresh what the world is every time. That's fine. The main thing you ought to do is generate a new systemd/dbus machine-id. But past this, I fairly frequently instantiate new systems by taking a btrfs snapshot of m…

> Windows would panic, certainly (because so much drivers & other state is persisted & expected)

Nope.

7B is because back in WinNT days it made sense to disable the drivers for which there are no devices in the system. Because memory and because people can't write drivers.

Nowadays it's AHCI or NVMe, which are pretty vendor agnostic, so you have 95% chance of successful boot. And if you boot is successful then Windows is fine and yes, it can grab the remaining drivers from the WU.

Re: Cloning a Laptop over NVMe TCP

#26

Post by _flux »

mrb wrote: post #5

In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…

> there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O

I guess most people don't have faster local network than an SSD can transfer.

I wonder though, for those people who do, does a concurrent I/O block device replicator tool exist?

Btw, you might want also use pv in the pipeline to see an ETA, although it might have a small impact on performance.

Re: Cloning a Laptop over NVMe TCP

#28

Post by bayindirh »

coretx wrote: post #27

Doing this without systemd or directly from the netboot environment would be interesting.

The user didn't do it from systemd actually. Instead they booted GRML, which is not very different from a netboot environment, and hand-exported the device themselves.

Re: Cloning a Laptop over NVMe TCP

#30

Post by baq »

If you directly connect devices over WiFi without an intermediate AP you should be able to double your transfer speed. In this scenario it might have been worth it.
Post reply on HN