Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GCP releases Spot VMs, the next generation of Pre-emptible VMs

cloud.google.com

Post reply on HN
Print view View original

71–80 of 129 posts

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#71

Post by tecleandor »

ur-whale wrote: post #55

So, probably a very naive question here, but how his the "continuity" across pre-emptions working? Say I am running a long batch process on one of these spot VMs and I get pre-empted ... Does my job restart where it was stopped automatically and everything is transparent save how long the job takes to complete, or do I have do to checkpointing myself and deal with the fact that my jobs may be killed at anytime? Also,…

Restarting the job is on you, the instance will be destroyed.

So you'll need to adapt your process to be resumable (or partially resumable through checkpoints) and/or idempotent, so nothing goes wrong if you run the job (or parts of the job) twice.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#72

Post by tecleandor »

superasn wrote: post #52
gravypod wrote: post #2

Any word on pricing?

Also what is the granularity? Right now on AWS you have to pay for the full hour even if you use it for 5 minutes. A per minute granularity can make Google's offering more enticing for a lot of users.

On AWS you only pay for full hours if you're using RHEL or Suse. Any other Linux distro or Windows pay by seconds:

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-int...

On GCP seems like you pay per seconds, but if you use a Premium OS and GCP stops your instance, you would pay for the premium OS pricing anyway:

https://cloud.google.com/compute/docs/instances/spot#spot-wi...

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#73

Post by Tenoke »

roca wrote: post #22
shock-value wrote: post #6

Earlier quoted context omitted.

It's probably a niche use case, but there is some utility to having a guaranteed daily shutoff. For example, you might spin up an instance as an on-demand remote dev environment, and the 24 hour cutoff ensures it doesn't accidentally get left on (over a weekend, for example). This would be easy to work around, but nonetheless could lead to unexpectedly high charges if you were relying on this behavior only to have it…

In AWS we just `sudo shutdown +1440`. Then we can cancel the shutdown later if we need to.

In AWS I set up a few lambdas to shut down my instances automatically because I've had some cases where everything crashes so thoroughly that the shutdown doesn't go off.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#74

Post by tecleandor »

ur-whale wrote: post #59
aetherspawn wrote: post #30

“requiredDuringSchedulingIgnoredDuringExecution” .. wow, what an epic length for a configuration option!

>“requiredDuringSchedulingIgnoredDuringExecution” .. wow, what an epic length for a configuration option Looks like Java culture is alive, well and spreading.

Seems Kubernetes-y:

https://kubernetes.io/docs/concepts/scheduling-eviction/assi...

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#75

Post by thundergolfer »

secondcoming wrote: post #69
bb101 wrote: post #64

Earlier quoted context omitted.

I find GCP's UI easier to use than AWS' by orders of magnitude.

I've found GCP's UI to be a racy mess. If you ask it to do something trivial like removing an instance from an Instance Group, it can show that it has done it when, in fact, it hasn't. Want to switch projects? Better remember to hit F5.

Yes, I've seen this myself. It's overall still miles better than AWS UI.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#76

Post by sitkack »

apendleton wrote: post #51
xyzzyz wrote: post #37

Earlier quoted context omitted.

Lack of spot VM means that all spare capacity has been taken up by the regular VMs, meaning that it spots are gone, so are regular ones.

Having worked on infrastructure deployed this way, I can say this oversimplifies a bit. There's virtually never no spot capacity at all, there's no capacity at a given bid price, for a given instance type, in a given region. You can start to scale up on-demand infrastructure using spot price as a signal, before there's literally none left, and you can also run on clusters of heterogeneous instances, and change your i…

No shade, but this is a very AWS specific view of cloud. Every cloud has a different resource allocation philosophy, not all the assumptions are the same.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#78

Post by londons_explore »

travisd wrote: post #3

I don’t see this in the article, but is there any reason why they’d introduce a new VM type rather than just removing the 24hour limit of preemptible VMs?

It smells like some person/team trying to get a promotion for launching a big new feature, rather than their perf packet saying "added option to not terminate after 24h".

Functionally, I see no difference.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#79

Post by deepsun »

secondcoming wrote: post #69
bb101 wrote: post #64

Earlier quoted context omitted.

I find GCP's UI easier to use than AWS' by orders of magnitude.

I've found GCP's UI to be a racy mess. If you ask it to do something trivial like removing an instance from an Instance Group, it can show that it has done it when, in fact, it hasn't. Want to switch projects? Better remember to hit F5.

How do you switch projects on AWS? In GCP it's a dropdown, in AWS it's completely different account (email or IAM or SAML backed) + captcha + MFA every time if not SAML.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#80

Post by jSherz »

deepsun wrote: post #79
secondcoming wrote: post #69

Earlier quoted context omitted.

I've found GCP's UI to be a racy mess. If you ask it to do something trivial like removing an instance from an Instance Group, it can show that it has done it when, in fact, it hasn't. Want to switch projects? Better remember to hit F5.

How do you switch projects on AWS? In GCP it's a dropdown, in AWS it's completely different account (email or IAM or SAML backed) + captcha + MFA every time if not SAML.

AWS SSO [1] negates a lot of that pain, but still won't let you have two accounts open in the same browser window. You can use email/password logins or a SAML source like Active Directory.

[1]: https://aws.amazon.com/single-sign-on/

Post reply on HN