Pipe with zstd (or similar)!
Cloning a Laptop over NVMe TCP
51–60 of 181 posts
Re: Cloning a Laptop over NVMe TCP
#52Earlier quoted context omitted.
NVMe/TCP or Clonezilla are vastly more moving parts and chances to mess up the options, compared to dd. In fact, the author's solution exposes his NVMe to unauthenticated remote write access by any number of clients(!) By comparison, the dd on the source is read-only, and the dd on the destination only accepts the first connection (yours) and no one else on the network can write to the disk. I strongly recommend agai…
> the author's solution exposes his NVMe to unauthenticated remote write access by any number of clients(!) I won't be bothered in a home network. > Clonezilla are vastly more moving parts ...and one of these moving parts is image integrity and write integrity verification, allowing byte-by-byte integrity during imaging and after write. > I strongly recommend against oflag=direct as in this... [snipped for brevity] U…
Your example proves me right. Your drive should be capable of 1000 MB/s but O_DIRECT reduces performance to 400 MB/s.
This matters in the specific use case of "netcat | gunzip | dd" as the compressed data rate on GigE will indeed be around 120 MB/s but when gunzip is decompressing unused parts of the filesystem (which compress very well), it will attempt to write 1+ GB/s or more to the pipe to dd and it would not be able to keep up with O_DIRECT.
Another thing you are doing wrong: benchmarking with /dev/zero. Many NVMe do transparent compression so writing zeroes is faster than writing random data and thus not a realistic benchmark.
PS: to clarify I am very well aware that not using O_DIRECT gives the impression initial writes are faster as they just fill the buffer cache. I am taking about sustained I/O performance over minutes as measured with, for example, iostat. You are talking to someone who has been doing Linux sysadmin and perf optimizations for 25 years :)
PPS: verifying data integrity is easy with the dd solution. I usually run "sha1sum /dev/nvme0nX" on both source and destination.
PPPS: I don't think Clonezilla is even capable of doing something similar (copying a remote disk to local disk without storing an intermediate disk image).
Re: Cloning a Laptop over NVMe TCP
#53Earlier quoted context omitted.
NVMe/TCP or Clonezilla are vastly more moving parts and chances to mess up the options, compared to dd. In fact, the author's solution exposes his NVMe to unauthenticated remote write access by any number of clients(!) By comparison, the dd on the source is read-only, and the dd on the destination only accepts the first connection (yours) and no one else on the network can write to the disk. I strongly recommend agai…
> the author's solution exposes his NVMe to unauthenticated remote write access by any number of clients(!) I won't be bothered in a home network. > Clonezilla are vastly more moving parts ...and one of these moving parts is image integrity and write integrity verification, allowing byte-by-byte integrity during imaging and after write. > I strongly recommend against oflag=direct as in this... [snipped for brevity] U…
> ...and one of these moving parts is image integrity and write integrity verification, allowing byte-by-byte integrity during imaging and after write.
dd followed by sha1sum on each end is still very few moving parts and should still be quite fast.
Re: Cloning a Laptop over NVMe TCP
#54In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…
Seems awesome. Can you please tell us how to use gzip or lz4 to do the imaging?
Re: Cloning a Laptop over NVMe TCP
#55In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…
Just cat the blockdev to a bash socket
Re: Cloning a Laptop over NVMe TCP
#56In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…
This is exactly that I usually do, it works like a charm
Re: Cloning a Laptop over NVMe TCP
#57This could also be titled: How to make one's life difficult 101 and/or I love experimenting.
I've bought a few hard disks in my lifetime. Many years ago one such disk came bundled with Acronis TrueImage. Back in the day you could just buy the darn thing, after certain year they switched it to subscription model. Anyhoo.. I got the "one-off" TrueImage and I've been using it ever since. I've 'migrated' the CD (physical mini-CD) it to a USB flash disk, and have been using it ever since.
I was curious for the solution as I recently bought a WinOS tablet, and would like to clone my PC to it, but this looks more like one too many hoop jumps to something that TrueImage can do in 2h hours (while watching a movie).
Re: Cloning a Laptop over NVMe TCP
#58Earlier quoted context omitted.
Isn't `nc -l -p 1234 > /dev/nvme0nX` working by accident (relying on that netcat is buffering its output in multiples of disk block size)?
No — the kernel buffers non-O_DIRECT writes to block devices to ensure correctness. Larger writes will be more efficient, however, if only due to reduced system call overhead. While not necessary when writing an image with the correct block size for the target device, even partial block overwrites work fine: # yes | head -c 512 > foo # losetup /dev/loop0 foo # echo 'Ham and jam and Spam a lot.' | dd bs=5 of=/dev/loop…
Somehow I had thought even in buffered mode the kernel would only accept block-aligned and sized I/O. TIL.
Re: Cloning a Laptop over NVMe TCP
#59Earlier quoted context omitted.
> the author's solution exposes his NVMe to unauthenticated remote write access by any number of clients(!) I won't be bothered in a home network. > Clonezilla are vastly more moving parts ...and one of these moving parts is image integrity and write integrity verification, allowing byte-by-byte integrity during imaging and after write. > I strongly recommend against oflag=direct as in this... [snipped for brevity] U…
Your example proves me right. Your drive should be capable of 1000 MB/s but O_DIRECT reduces performance to 400 MB/s. This matters in the specific use case of "netcat | gunzip | dd" as the compressed data rate on GigE will indeed be around 120 MB/s but when gunzip is decompressing unused parts of the filesystem (which compress very well), it will attempt to write 1+ GB/s or more to the pipe to dd and it would not be…
> Your example proves me right. Your drive should be capable of 1000 MB/s but O_DIRECT reduces performance to 400 MB/s.
I noted that the bus I connected the device has 500MBps bandwidth theoretical, no?
To cite myself:
> Target is a Samsung T7 Shield 2TB, with 1050MB/sec sustained write speed. Bus is USB 3.0 with 500MBps top speed (so I can go %50 of drive speeds). Result is 404MBps, which is fair for the bus.
Re: Cloning a Laptop over NVMe TCP
#60Earlier quoted context omitted.
> the author's solution exposes his NVMe to unauthenticated remote write access by any number of clients(!) I won't be bothered in a home network. > Clonezilla are vastly more moving parts ...and one of these moving parts is image integrity and write integrity verification, allowing byte-by-byte integrity during imaging and after write. > I strongly recommend against oflag=direct as in this... [snipped for brevity] U…
> ...and one of these moving parts is image integrity and write integrity verification, allowing byte-by-byte integrity during imaging and after write. dd followed by sha1sum on each end is still very few moving parts and should still be quite fast.
Yes, in the laptop and one-off case, that's true.
In a data center it's not (this is when I use clonezilla 99.9% of the time, tbf).