Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Preemptible VMs

googlecloudplatform.blogspot.com

Post reply on HN
Print view View original

31–40 of 98 posts

Re: Introducing Preemptible VMs

#31

Post by CoolGuySteve »

vosper wrote: post #18
ikeboy wrote: post #4

How does their pricing compare to ec2 spot instances?

The fundamental difference seems to be that Google's prices are fixed, whereas AWS uses a "market" model, which frequently sees crazy prices (well over the on-demand price) especially in us-east-1. I say "market" because no-one really knows how the spot market place actually works. We've had machines run for weeks, and other times the prices fluctuate in bizarre ways and we can't get out preferred instance types for…

We had the same problem with getting priced out of c3.8xlarge in Virginia. We fixed it by changing our allocation algorithm to find alternate instance types and zones. For example, instead of 1 c3.8xlarge, it might pick 2 c3.4xlarge instances or a cc2.8xlarge. Seems to work so far.

I looked through the pricing table and played with the calculator, it seems something equivalent to our needs would cost around a third more on google but each cpu would have twice as much ram. Not worth it for us.

Re: Introducing Preemptible VMs

#32

Post by pacala »

boulos wrote: post #3
gnur wrote: post #2

I wonder how big the chance is that the machine is turned off in the first hour. I use Google Compute for some personal project and the typical run time for a VM is about 10 - 20 minutes. If the vm has 90% chance to survive the first hour, it could be worth the trouble to make my process more fault tolerable.

We're not ready to put hard numbers on it. As our docs ( https://cloud.google.com/compute/docs/instances/preemptible ) say: The probability that Compute Engine will terminate a preemptible instance for a system event is generally low, but may vary from day to day and from zone to zone depending on current conditions. Give it a shot in us-central1-a and let us know how it goes! [Edit to undo my quote text (it wrapped…

I couldn't figure out from the docs, can one restart a preempted preemptible instance, or needs to start a different one? Would it be possible to restart in non-preemptible mode, so the job completes but at a higher price? We still want to complete our workloads, one way or another :)

Re: Introducing Preemptible VMs

#33

Post by comex »

CydeWeys wrote: post #27
andybak wrote: post #15

Any reason you couldn't use these for conventional web stuff? 30 seconds could easily be long enough to bring up another instance and sync data if needed.

By "conventional web stuff", do you mean serving a website or similar? Yeah, I don't think it makes sense to host a website on a service that is preemptible. You need your webserver available to respond to requests 24/7. It can't just randomly go down. Pre-emptible instances are more suited for batch processing and big data computation runs.

If you have a website that can be distributed among arbitrarily many frontend servers, it could make sense to put them on preemptible instances, with the database on a regular instance. (In the unlikely case no preemptible instances are available, you could always automatically switch the frontend servers to regular ones.) However, I think this could only possibly be efficient pricing-wise if your traffic was extremely bursty.

Re: Introducing Preemptible VMs

#34

Post by spankalee »

andybak wrote: post #15

Any reason you couldn't use these for conventional web stuff? 30 seconds could easily be long enough to bring up another instance and sync data if needed.

No reason at all. If you have enough preemptible VMs, and they spin up fast enough, then when one goes down, just spin up another. If for some reason GCE's overall preemptible spare capacity drops and you can't get a new one and it's important to you, then spin up a regular VM.

Re: Introducing Preemptible VMs

#35

Post by h43k3r »

Now I am waiting to see a similar thing in Azure.

Preemptible VMs will prove to be very useful in fault tolerant Distributed Networks.

One more usecase that I can think off is Load Testing on a large scale ( in a distributed way ).

Re: Introducing Preemptible VMs

#36

Post by boulos »

pacala wrote: post #32
boulos wrote: post #3

Earlier quoted context omitted.

We're not ready to put hard numbers on it. As our docs ( https://cloud.google.com/compute/docs/instances/preemptible ) say: The probability that Compute Engine will terminate a preemptible instance for a system event is generally low, but may vary from day to day and from zone to zone depending on current conditions. Give it a shot in us-central1-a and let us know how it goes! [Edit to undo my quote text (it wrapped…

I couldn't figure out from the docs, can one restart a preempted preemptible instance, or needs to start a different one? Would it be possible to restart in non-preemptible mode, so the job completes but at a higher price? We still want to complete our workloads, one way or another :)

Yes you can start a VM back up via gcloud:

   gcloud compute instances start instance-name
Note of course that if you got preempted because we needed the capacity back for regular VMs (as opposed to say a maintenance event) you may not be able to start a new Preemptible VM in that zone.

Re: Introducing Preemptible VMs

#37

Post by ErikRogneby »

andybak wrote: post #15

Any reason you couldn't use these for conventional web stuff? 30 seconds could easily be long enough to bring up another instance and sync data if needed.

I would say it depends on the size of your pool. 30 seconds _is not_ long enough to bring up another instance.

Re: Introducing Preemptible VMs

#38

Post by boulos »

CoolGuySteve wrote: post #31
vosper wrote: post #18

Earlier quoted context omitted.

The fundamental difference seems to be that Google's prices are fixed, whereas AWS uses a "market" model, which frequently sees crazy prices (well over the on-demand price) especially in us-east-1. I say "market" because no-one really knows how the spot market place actually works. We've had machines run for weeks, and other times the prices fluctuate in bizarre ways and we can't get out preferred instance types for…

We had the same problem with getting priced out of c3.8xlarge in Virginia. We fixed it by changing our allocation algorithm to find alternate instance types and zones. For example, instead of 1 c3.8xlarge, it might pick 2 c3.4xlarge instances or a cc2.8xlarge. Seems to work so far. I looked through the pricing table and played with the calculator, it seems something equivalent to our needs would cost around a third m…

For people coming from AWS, we currently don't have an instance shape that lines up with the c3/c4 ratio (pushing you either to our n1-standards or n1-highcpu-. Can you get by with less memory?

Note: we're very aware of this pain point, and maybe you'll see something soon ;).

Re: Introducing Preemptible VMs

#39

Post by CoolGuySteve »

boulos wrote: post #38
CoolGuySteve wrote: post #31

Earlier quoted context omitted.

We had the same problem with getting priced out of c3.8xlarge in Virginia. We fixed it by changing our allocation algorithm to find alternate instance types and zones. For example, instead of 1 c3.8xlarge, it might pick 2 c3.4xlarge instances or a cc2.8xlarge. Seems to work so far. I looked through the pricing table and played with the calculator, it seems something equivalent to our needs would cost around a third m…

For people coming from AWS, we currently don't have an instance shape that lines up with the c3/c4 ratio (pushing you either to our n1-standards or n1-highcpu-. Can you get by with less memory? Note: we're very aware of this pain point, and maybe you'll see something soon ;).

We need maybe 1GB for each job, but we need some leeway to avoid the OOM killer when a group of work units is larger. Swapping takes so long that it's not cost effective.

But to be honest about the situation, the cost would have to be much lower to make it worthwhile for me to rewrite our scheduler on Google's API.

Post reply on HN