Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloning a Laptop over NVMe TCP

copyninja.in

Post reply on HN
Print view View original

131–140 of 181 posts

Re: Cloning a Laptop over NVMe TCP

#131

Post by wolrah »

crote wrote: post #105
_flux wrote: post #26

Earlier quoted context omitted.

> there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O I guess most people don't have faster local network than an SSD can transfer. I wonder though, for those people who do, does a concurrent I/O block device replicator tool exist? Btw, you might want also use pv in the pipeline to see an ETA, although it might have a small impact o…

I doubt it makes a difference. SSDs are an awful lot better at sequential writes than random writes, and concurrent IO would mainly speed up random access. Besides, I don't think anyone really has a local network which is faster than their SSD. Even a 4-year-old consumer Samsung 970 Pro can sustain full-disk writes at 2.000M Byte/s, easily saturating a 10Gbit connection. If we're looking at state-of-the-art consumer…

> Besides, I don't think anyone really has a local network which is faster than their SSD. Even a 4-year-old consumer Samsung 970 Pro can sustain full-disk writes at 2.000M Byte/s, easily saturating a 10Gbit connection.

You might be surprised if you take a look at how cheap high speed NICs are on the used market. 25G and 40G can be had for around $50, and 100G around $100. If you need switches things start to get expensive but for the "home lab" crowd since most of these cards are dual port a three-node mesh can be had for just a few hundred bucks. I've had a 40G link to my home server for a few years now mostly just because I could do it for less than the cost of a single hard drive.

Re: Cloning a Laptop over NVMe TCP

#132

Post by loeg »

mrb wrote: post #5

In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…

Agree, but I'd suggest zstd instead of gzip (or lz4 is fine).

Re: Cloning a Laptop over NVMe TCP

#133

Post by mixmastamyk »

dmos62 wrote: post #122
maccard wrote: post #75

Earlier quoted context omitted.

> I wonder what could I have done better. 6 hours is roughly 10MB/s, so you likely could have gone much much quicker. Did you compress with `-z`? If you could use ethernetyou probably could have done it at closer to 100MB/s on most deviceds, which would have been 35 minutes.

No, I didn't use compression. Would it be useful over a high-bandwidth connection? I presume that it wasn't wifi bandwidth that was bottlenecking, though I've not really checked. One thing I could have done is found a way to track total progress, so that I could have noticed that this is going way too slow.

One of the options of rsync is to print out transfer speed, --progress or verbose or similar.

Re: Cloning a Laptop over NVMe TCP

#134

Post by baq »

greenshackle2 wrote: post #125
vetinari wrote: post #120

Earlier quoted context omitted.

They would receive APIPA (169.254.0.0/16) address for IPv4 and link-local for IPv6, if the interface would be brought up. Now, that second part is a question; windows would do it, but linux probably not.

If both ends got APIPA addresses would they be able to talk to each other? I was under the impression you have to set up the devices as each other's default gateway, but maybe I'm the one not up to modern standards this time.

There's https://en.wikipedia.org/wiki/Multicast_DNS though I'll admit I never properly tried it.

Re: Cloning a Laptop over NVMe TCP

#135

Post by PeterisP »

JoeAltmaier wrote: post #107
crote wrote: post #105

Earlier quoted context omitted.

I doubt it makes a difference. SSDs are an awful lot better at sequential writes than random writes, and concurrent IO would mainly speed up random access. Besides, I don't think anyone really has a local network which is faster than their SSD. Even a 4-year-old consumer Samsung 970 Pro can sustain full-disk writes at 2.000M Byte/s, easily saturating a 10Gbit connection. If we're looking at state-of-the-art consumer…

Can't find corroboration for the assertion 'SSDs are an awful lot better at sequential writes than random writes'. Doesn't make sense at first glance. There's no head to move, as in an old-style hard drive. What else could make random write take longer on an SSD?

The key aspect is that such memory generally works on a "block" level so making any smaller-than-block write on a SSD requires reading a whole block (which can be quite large), erasing that whole block and then writing back the whole modified block; as you physically can't toggle a bit without erasing the whole block first.

So if large sequential writes mean that you only write full whole blocks, that can be done much faster than writing the same data in random order.

Re: Cloning a Laptop over NVMe TCP

#136

Post by neuromanser »

jasomill wrote: post #49
dezgeg wrote: post #39

Earlier quoted context omitted.

Isn't `nc -l -p 1234 > /dev/nvme0nX` working by accident (relying on that netcat is buffering its output in multiples of disk block size)?

No — the kernel buffers non-O_DIRECT writes to block devices to ensure correctness. Larger writes will be more efficient, however, if only due to reduced system call overhead. While not necessary when writing an image with the correct block size for the target device, even partial block overwrites work fine: # yes | head -c 512 > foo # losetup /dev/loop0 foo # echo 'Ham and jam and Spam a lot.' | dd bs=5 of=/dev/loop…

> # yes | head -c 512 > foo

How about `truncate -s 512 foo`?

Re: Cloning a Laptop over NVMe TCP

#137

Post by adrian_b »

greenshackle2 wrote: post #125
vetinari wrote: post #120

Earlier quoted context omitted.

They would receive APIPA (169.254.0.0/16) address for IPv4 and link-local for IPv6, if the interface would be brought up. Now, that second part is a question; windows would do it, but linux probably not.

If both ends got APIPA addresses would they be able to talk to each other? I was under the impression you have to set up the devices as each other's default gateway, but maybe I'm the one not up to modern standards this time.

Their randomly chosen addresses are within the same /16 subnetwork.

Therefore they can talk directly, without using a gateway.

Their corresponding Ethernet MAC addresses will be resolved by ARP.

The problem is that in many cases you would have to look at the autoconfigured addresses and introduce manually the peer address in each computer.

For file sharing, either in Windows or using Samba on Linux, you could autodiscover the other computer and just use the name of the shared resource.

Re: Cloning a Laptop over NVMe TCP

#138

Post by BeeOnRope »

crote wrote: post #105
_flux wrote: post #26

Earlier quoted context omitted.

> there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O I guess most people don't have faster local network than an SSD can transfer. I wonder though, for those people who do, does a concurrent I/O block device replicator tool exist? Btw, you might want also use pv in the pipeline to see an ETA, although it might have a small impact o…

I doubt it makes a difference. SSDs are an awful lot better at sequential writes than random writes, and concurrent IO would mainly speed up random access. Besides, I don't think anyone really has a local network which is faster than their SSD. Even a 4-year-old consumer Samsung 970 Pro can sustain full-disk writes at 2.000M Byte/s, easily saturating a 10Gbit connection. If we're looking at state-of-the-art consumer…

In EC2, most of the "storage optimized" instances (which have the largest/fastest SSDs) generally have more advertised network throughput than SSD throughput, by a factor usually in the range of 1 to 2 (though it depends on exactly how you count it, e.g., how you normalize for the full-duplex nature of network speeds and same for SSD).

Re: Cloning a Laptop over NVMe TCP

#139

Post by tristor »

This is very clever, although somewhat unnecessary, but still useful to know. The one thing I would call out is the author used WiFi because one of the laptops didn't have Ethernet. I've encountered this situation several times myself and I've found that nearly every modern laptop supports USB in both modes, so you can simply use a USB-C cable to connect the two laptops and get a pseudo-Ethernet device this way to interconnect them, no need to use WiFi. This is hundreds of times faster than WiFi.

Re: Cloning a Laptop over NVMe TCP

#140

Post by rwmj »

verticalscaler wrote: post #96
throwawaaarrgh wrote: post #85

Earlier quoted context omitted.

This is actually much better because nbdcopy can handle sparse files, you can set the number of connections and threads to number of cores, you can force a flush before exit, and enable a progress bar. For unencrypted drives it also supports TLS.

If you really want a progress bar chuck in a 'pv' somewhere into the command posted at the top of the thread.

Or add nbdcopy -p option :-)
Post reply on HN