Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloning a Laptop over NVMe TCP

copyninja.in

Post reply on HN
Print view View original

101–110 of 181 posts

Re: Cloning a Laptop over NVMe TCP

#101

Post by [deleted] »

fransje26 wrote: post #98
mrb wrote: post #5

In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…

I am not often speechless, but this hit the spot. Well done Sir! Where does one learn this black art?

[deleted]

Re: Cloning a Laptop over NVMe TCP

#102

Post by greenshackle2 »

roomey wrote: post #83
adrian_b wrote: post #38

Earlier quoted context omitted.

I assume that those USB-C connectors were USB 4 or Thunderbolt, not USB 3. With Thunderbolt and operating systems that support Ethernet over Thunderbolt, a virtual network adapter is automatically configured for any Thunderbolt connector, so connecting a USB C cable between 2 such computers should just work, as if they had Ethernet 10 Gb/s connectors. With USB 3 USB C connectors, you must use USB network adapters (up…

Yes, both were modern enough laptops. Although the ideapad didn't advertise thunderbolt in the lspci, connecting that and the dell precision "just worked" (tm). It's very useful for sending large data using minimal equipment. No need for two cat6 cables and a router for example.

You just need a single Ethernet cable really, if the devices are reasonably modern. With Auto MDI-X the days of needing a crossover cable or a switch are over.

Re: Cloning a Laptop over NVMe TCP

#103

Post by fransje26 »

Gabrys1 wrote: post #9

I usually set up an initial distro and then copy over my /home. Later, I just need to install the debs I'm missing, but this has the benefit of not installing stuff I don't need anymore. That said, I didn't know you could export NVMe over TCP like that, so still a nice read!

Same here.

The only problem with that approach is that it is also copies over the .config and .cache folders, most of which are possibly not needed anymore. Or worse, they might contain configs that overrides better/newer parameters set by the new system..

Re: Cloning a Laptop over NVMe TCP

#104

Post by 1970-01-01 »

jurgenkesker wrote: post #76

Or just use Clonezilla? Then you also copy only the actual data blocks, and it can autoresize your partitions as well. That's how I always do it. True, I do always just take the NVME disk out of the laptop and put it in a highspeed dock.

Clonezilla is great. It's got one job and it usually succeeds the first time. My only complaint is the initial learning curve requires tinkering. It's still not at the trust level of fire and forget. Experimenting is recommended as backup is never the same thing as a backup and restore and even Clonezilla will have issues recreating partitions on disks that are very different from their source.

Re: Cloning a Laptop over NVMe TCP

#105

Post by crote »

_flux wrote: post #26
mrb wrote: post #5

In the author's scenario, there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O. All the complex commands can be replaced by a simple netcat. On the destination laptop: $ nc -l -p 1234 | dd of=/dev/nvme0nX bs=1M On the source laptop: $ nc x.x.x.x 1234 The dd on the destination is just to buffer writes so they are faster/more efficient…

> there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O I guess most people don't have faster local network than an SSD can transfer. I wonder though, for those people who do, does a concurrent I/O block device replicator tool exist? Btw, you might want also use pv in the pipeline to see an ETA, although it might have a small impact o…

I doubt it makes a difference. SSDs are an awful lot better at sequential writes than random writes, and concurrent IO would mainly speed up random access.

Besides, I don't think anyone really has a local network which is faster than their SSD. Even a 4-year-old consumer Samsung 970 Pro can sustain full-disk writes at 2.000M Byte/s, easily saturating a 10Gbit connection.

If we're looking at state-of-the-art consumer tech, the fastest you're getting is a USB4 40Gbit machine-to-machine transfer - but at that point you probably have something like the Crucial T700, which has a sequential write speed of 11.800 MByte/s.

The enterprise world probably doesn't look too different. You'd need a 100Gbit NIC to saturate even a single modern SSD, but any machine with such a NIC is more likely to have closer to half a dozen SSDs. At that point you're starting to be more worried about things like memory bandwidth instead. [0]

[0]: http://nabstreamingsummit.com/wp-content/uploads/2022/05/202...

Re: Cloning a Laptop over NVMe TCP

#106

Post by greenshackle2 »

HenryBemis wrote: post #57

This could also be titled: How to make one's life difficult 101 and/or I love experimenting. I've bought a few hard disks in my lifetime. Many years ago one such disk came bundled with Acronis TrueImage. Back in the day you could just buy the darn thing, after certain year they switched it to subscription model. Anyhoo.. I got the "one-off" TrueImage and I've been using it ever since. I've 'migrated' the CD (physical…

Next time, I'll be sure to do it the easy way by going back in time and getting a free copy of Acronis TrueImage in 1998.

Re: Cloning a Laptop over NVMe TCP

#107

Post by JoeAltmaier »

crote wrote: post #105
_flux wrote: post #26

Earlier quoted context omitted.

> there are zero benefits in using NVMe/TCP, as he just ends up doing a serial block copy using dd(1) so he's not leveraging concurrent I/O I guess most people don't have faster local network than an SSD can transfer. I wonder though, for those people who do, does a concurrent I/O block device replicator tool exist? Btw, you might want also use pv in the pipeline to see an ETA, although it might have a small impact o…

I doubt it makes a difference. SSDs are an awful lot better at sequential writes than random writes, and concurrent IO would mainly speed up random access. Besides, I don't think anyone really has a local network which is faster than their SSD. Even a 4-year-old consumer Samsung 970 Pro can sustain full-disk writes at 2.000M Byte/s, easily saturating a 10Gbit connection. If we're looking at state-of-the-art consumer…

Can't find corroboration for the assertion 'SSDs are an awful lot better at sequential writes than random writes'.

Doesn't make sense at first glance. There's no head to move, as in an old-style hard drive. What else could make random write take longer on an SSD?

Re: Cloning a Laptop over NVMe TCP

#108

Post by roomey »

greenshackle2 wrote: post #102
roomey wrote: post #83

Earlier quoted context omitted.

Yes, both were modern enough laptops. Although the ideapad didn't advertise thunderbolt in the lspci, connecting that and the dell precision "just worked" (tm). It's very useful for sending large data using minimal equipment. No need for two cat6 cables and a router for example.

You just need a single Ethernet cable really, if the devices are reasonably modern. With Auto MDI-X the days of needing a crossover cable or a switch are over.

I'm not sure, first off the precision doesn't have an ethernet port at all!

Secondly, I'm not sure if a crossover cable setup will autoconfigure the network, as the poster above says, it has been since the 90s when I bothered trying something like that!

Re: Cloning a Laptop over NVMe TCP

#109

Post by crote »

mrb wrote: post #64
bayindirh wrote: post #59

Earlier quoted context omitted.

> Your example proves me right. Your drive should be capable of 1000 MB/s but O_DIRECT reduces performance to 400 MB/s. I noted that the bus I connected the device has 500MBps bandwidth theoretical, no? To cite myself: > Target is a Samsung T7 Shield 2TB, with 1050MB/sec sustained write speed. Bus is USB 3.0 with 500MBps top speed (so I can go %50 of drive speeds). Result is 404MBps, which is fair for the bus.

Yes USB3.0 is 500 MB/s but are you sure your bus is 3.0? It would imply your machine is 10+ years old. Most likely it's 3.1 or newer which is 1000 MB/s. And again, benchmarking /dev/zero is invalid anyway as I explained (transparent compression)

No, it wouldn't imply the machine is 10+ years old. Even a state-of-the-art motherboard like the Gigabyte Z790 D AX (which became available in my country today) has more USB 3 gen1 (5Gbps) ports than gen2 (10Gbps).

The 5Gbps ports are just marketed as "USB 3.1" instead of "USB 3.0" these days, because USB naming is confusing and the important part is the "gen x".

Re: Cloning a Laptop over NVMe TCP

#110

Post by wtallis »

JoeAltmaier wrote: post #107
crote wrote: post #105

Earlier quoted context omitted.

I doubt it makes a difference. SSDs are an awful lot better at sequential writes than random writes, and concurrent IO would mainly speed up random access. Besides, I don't think anyone really has a local network which is faster than their SSD. Even a 4-year-old consumer Samsung 970 Pro can sustain full-disk writes at 2.000M Byte/s, easily saturating a 10Gbit connection. If we're looking at state-of-the-art consumer…

Can't find corroboration for the assertion 'SSDs are an awful lot better at sequential writes than random writes'. Doesn't make sense at first glance. There's no head to move, as in an old-style hard drive. What else could make random write take longer on an SSD?

The main problem is that random writes tend to be smaller than the NAND flash erase block size, which is in the range of several MB.

You can check literally any SSD benchmark that tests both random and sequential IO. They're both vastly better than a mechanical hard drive, but sequential IO is still faster than random IO.

Post reply on HN