41–50 of 181 posts
Earlier quoted context omitted.
If you moved the wrong stuff Linux would give you a bad time too, try /proc, /dev
Those are pseudo-filesystems though, they aren't part of the install.
Edit: or backup process
Earlier quoted context omitted.
I don't see how you can consider the nvme over tcp version less moving parts. dd is installed on every system, and if you don't have nc you can still use ssh and sacrifice a bit of performance. dd if=/dev/foo | ssh dest@bar "cat > /dev/moo"
NVMe over TCP encapsulates and shows me the remote device as is. Just a block device. I just copy that block device with "dd", that's all. It's just a dumb pipe encapsulated with TCP, which is already battle tested enough. Moreover, if I have fatter pipe, I can tune dd for better performance with a single command.
Earlier quoted context omitted.
This use of dd may cause corruption! You need iflag=fullblock to ensure it doesn't truncate any blocks, and (at the risk of cargo-culting) conv=sync doesn't hurt as well. I prefer to just nc -l -p 1234 > /dev/nvme0nX.
According to the documentation of dd, "iflag=fullblock" is required only when dd is used with the "count=" option. Otherwise, i.e. when dd has to read the entire input file because there is no "count=" option, "iflag=fullblock" does not have any documented effect. From "info dd": "If short reads occur, as could be the case when reading from a pipe for example, ‘iflag=fullblock’ ensures that ‘count=’ counts complete i…
In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…
This use of dd may cause corruption! You need iflag=fullblock to ensure it doesn't truncate any blocks, and (at the risk of cargo-culting) conv=sync doesn't hurt as well. I prefer to just nc -l -p 1234 > /dev/nvme0nX.
Earlier quoted context omitted.
As a sysadmin, I'd rather use NVMe TCP or Clonezilla to do a slow write rather than trying to go 5% faster with more moving parts and chance to corrupt my drive in the process. Plus, a it'd be well deserved coffee break. Considering I'd be going at GigE speeds at best, I'd add "oflag=direct" to bypass caching on the target. A bog standard NVMe can write >300MBps unhindered, so trying to cache is moot. Lastly, parted…
NVMe/TCP or Clonezilla are vastly more moving parts and chances to mess up the options, compared to dd. In fact, the author's solution exposes his NVMe to unauthenticated remote write access by any number of clients(!) By comparison, the dd on the source is read-only, and the dd on the destination only accepts the first connection (yours) and no one else on the network can write to the disk. I strongly recommend agai…
I won't be bothered in a home network.
> Clonezilla are vastly more moving parts
...and one of these moving parts is image integrity and write integrity verification, allowing byte-by-byte integrity during imaging and after write.
> I strongly recommend against oflag=direct as in this... [snipped for brevity]
Unless you're getting a bottom of the barrel NVMe, all of them have DRAM caches and do their own write caching independent of O_DIRECT, which only bypasses OS caches. Unless the pipe you have has higher throughput than your drive, caching in the storage device's controller ensures optimal write speeds.
I can hit theoretical maximum write speeds of all my SSDs (internal or external) with O_DIRECT. When the pipe is fatter or the device can't sustain that speeds, things go south, but this is why we have knobs.
When you don't use O_DIRECT in these cases, you see initial speed surge maybe, but total time doesn't reduce.
TL;DR: When you're getting your data at 100MBps at most, using O_DIRECT on an SSD with 1GBps write speeds doesn't affect anything. You're not saturating anything on the pipe.
Just did a small test:
dd if=/dev/zero of=test.file bs=1024kB count=3072 oflag=direct status=progress
2821120000 bytes (2.8 GB, 2.6 GiB) copied, 7 s, 403 MB/s
3072+0 records in
3072+0 records out
3145728000 bytes (3.1 GB, 2.9 GiB) copied, 7.79274 s, 404 MB/s
If the drive didn't have its own cache, caching on the OS side would have more profound effect since I can queue more writes to device and pool them at RAM.
In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…
Earlier quoted context omitted.
This use of dd may cause corruption! You need iflag=fullblock to ensure it doesn't truncate any blocks, and (at the risk of cargo-culting) conv=sync doesn't hurt as well. I prefer to just nc -l -p 1234 > /dev/nvme0nX.
Isn't `nc -l -p 1234 > /dev/nvme0nX` working by accident (relying on that netcat is buffering its output in multiples of disk block size)?
Earlier quoted context omitted.
This use of dd may cause corruption! You need iflag=fullblock to ensure it doesn't truncate any blocks, and (at the risk of cargo-culting) conv=sync doesn't hurt as well. I prefer to just nc -l -p 1234 > /dev/nvme0nX.
Isn't `nc -l -p 1234 > /dev/nvme0nX` working by accident (relying on that netcat is buffering its output in multiples of disk block size)?
Larger writes will be more efficient, however, if only due to reduced system call overhead.
While not necessary when writing an image with the correct block size for the target device, even partial block overwrites work fine:
# yes | head -c 512 > foo
# losetup /dev/loop0 foo
# echo 'Ham and jam and Spam a lot.' | dd bs=5 of=/dev/loop0
5+1 records in
5+1 records out
28 bytes copied, 0.000481667 s, 58.1 kB/s
# hexdump -C /dev/loop0
00000000 48 61 6d 20 61 6e 64 20 6a 61 6d 20 61 6e 64 20 |Ham and jam and |
00000010 53 70 61 6d 20 61 20 6c 6f 74 2e 0a 79 0a 79 0a |Spam a lot..y.y.|
00000020 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a 79 0a |y.y.y.y.y.y.y.y.|
*
00000200
Finally, note that this applies to most block devices, but tape devices work differently: partial overwrites are not supported, and, in variable block mode, the size of individual write calls determines the resulting tape block sizes.
How can this work if the laptops have different hardware, cq different requirements on device drivers?