61–70 of 245 posts
Heard from a Googler that the internal infrastructure (Borg) is simply not optimized for quick startup. Launching a new Borg job often takes multiple minutes before the job runs. Not surprising at all.
booting VMs != starting a borg job.
Earlier quoted context omitted.
How does migrating a vm to another physical machine work?
VMware has been doing this for years, it's called vmotion and there is a lot of documentation about it if you are interested (eg https://www.thegeekpub.com/8407/how-vmotion-works/ ) Essential, memory state is copied to the new host, the VM is stunned for a millisecond and the cpu states is copied and resumed on the new host (you may see a dropped ping). All the networking and storage is virtual anyway so that is "mov…
Very cool.
Earlier quoted context omitted.
Assuming this blurb is accurate: " General-purpose SSD volume (gp3) provides the consistent 125 MiB/s throughput and 3000 IOPS within the price of provisioned storage. Additional IOPS (up to 16,000) and throughput (1000 MiB/s) can be provisioned with an additional price. The General-purpose SSD volume (gp2) provides 3 IOPS per GiB storage provisioned with a minimum of 100 IOPS" ... then it seems like a device that li…
This would have been my guess. All EBS volumes are stored on a physical disk that supports the highest bandwidth and IOPS you can live migrate to, and the actual rates you get are determined by something in the interconnect. Live migration is thus a matter of swapping out the interconnect between the VM and the disk or even just relaxing a logical rate-limiter, without having to migrate your data to a different disk.
Earlier quoted context omitted.
They pause your VM, copy everything about its state over to the new machine, and quickly start the other instance. It's pretty clever. I think there are tricks you can play with machines that have large memory footprints to copy most of it before the pause, and only copy what has changed since then during the pause. The disks are all on the network, so no need to move anything there.
In reality it sync the memory first to the other host and only pause the vm when the last state sync is small enough to be so quick the pause is barely measurable.
Earlier quoted context omitted.
How does migrating a vm to another physical machine work?
VMware has been doing this for years, it's called vmotion and there is a lot of documentation about it if you are interested (eg https://www.thegeekpub.com/8407/how-vmotion-works/ ) Essential, memory state is copied to the new host, the VM is stunned for a millisecond and the cpu states is copied and resumed on the new host (you may see a dropped ping). All the networking and storage is virtual anyway so that is "mov…
Earlier quoted context omitted.
Dunno about AWS, but GCP uses live migration, and will migrate your VM across physical machines as necessary. The disk volumes are all connected over the network, nothing really depends on the actual physical machine your VM is ran on.
Azure, AWS and GCP all have live migration. VMWare has it too.
Heard from a Googler that the internal infrastructure (Borg) is simply not optimized for quick startup. Launching a new Borg job often takes multiple minutes before the job runs. Not surprising at all.
Borg is not used for gcp vms, though.
Earlier quoted context omitted.
How does migrating a vm to another physical machine work?
VMware has been doing this for years, it's called vmotion and there is a lot of documentation about it if you are interested (eg https://www.thegeekpub.com/8407/how-vmotion-works/ ) Essential, memory state is copied to the new host, the VM is stunned for a millisecond and the cpu states is copied and resumed on the new host (you may see a dropped ping). All the networking and storage is virtual anyway so that is "mov…