Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GCP releases Spot VMs, the next generation of Pre-emptible VMs

cloud.google.com

Post reply on HN
Print view View original

21–30 of 129 posts

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#21

Post by jiggawatts »

titzer wrote: post #16
wmf wrote: post #10

Earlier quoted context omitted.

You might be misunderstanding how spot works. The VM either runs at full speed or not at all so you know what you're getting.

Nevertheless, doesn't that force you to divide your work up into units somehow and manage external storage to track what/how much is done?

Yes, but this is common for "cloud native" applications. Web servers are ideally stateless "cattle", not "pets". Code is pulled down via CI/CD pipelines or via container images. State is in databases or blobs. Logs stream to some sort of analytics system.

Similarly, Kubernetes works well with spot VMs. You can have Spot and Regular node pools in one cluster, and then your workloads can happily spread out to the extra capacity at a huge discount.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#22

Post by roca »

shock-value wrote: post #6
travisd wrote: post #3

I don’t see this in the article, but is there any reason why they’d introduce a new VM type rather than just removing the 24hour limit of preemptible VMs?

It's probably a niche use case, but there is some utility to having a guaranteed daily shutoff. For example, you might spin up an instance as an on-demand remote dev environment, and the 24 hour cutoff ensures it doesn't accidentally get left on (over a weekend, for example). This would be easy to work around, but nonetheless could lead to unexpectedly high charges if you were relying on this behavior only to have it…

In AWS we just `sudo shutdown +1440`. Then we can cancel the shutdown later if we need to.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#23

Post by wbl »

titzer wrote: post #16
wmf wrote: post #10

Earlier quoted context omitted.

You might be misunderstanding how spot works. The VM either runs at full speed or not at all so you know what you're getting.

Nevertheless, doesn't that force you to divide your work up into units somehow and manage external storage to track what/how much is done?

You have to do that anyway for serious batch processing.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#24

Post by toast0 »

titzer wrote: post #7

I have trouble thinking of a use case that fits this VM type. Is it for batch processing tasks that don't have tight deadlines? The lack of any real guarantees makes it hard to price, as you have no idea what you're actually paying for. Would it make sense to have a model that includes deadlines in its pricing (like give this process 2.5 hours of CPU and 4GB of RAM to complete this task by...next Tuesday)?

The 30 second shutdown warning makes this a bit tricky for interactive work, but if your interactive requests will all finish within 20 seconds or so (or you can hand them off), and 5 seconds to get out of load balancing, this could be cost savings for whatever fits; some base number of nodes + a number of spot nodes as demand requires, if no spot nodes are available, serve from somewhat overloaded base nodes + maybe some regular priced nodes (but fewer extra nodes than at spot prices).

Or, more likely, for batch processing that's mostly time insensitive. Things that can be resumed without losing much work or lots of small short jobs. Retranscoding media with new settings comes to mind. If you've got deadlines, you'd probably need a mix of on-demand and spot nodes.

This is the corner of cloud pricing where if you can optimize use of this, you might be able to do better than traditional hosting. But only if your needs are variable, or you get a lot of high discount spot pricing.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#25

Post by cma »

shock-value wrote: post #6
travisd wrote: post #3

I don’t see this in the article, but is there any reason why they’d introduce a new VM type rather than just removing the 24hour limit of preemptible VMs?

It's probably a niche use case, but there is some utility to having a guaranteed daily shutoff. For example, you might spin up an instance as an on-demand remote dev environment, and the 24 hour cutoff ensures it doesn't accidentally get left on (over a weekend, for example). This would be easy to work around, but nonetheless could lead to unexpectedly high charges if you were relying on this behavior only to have it…

Tha sounds like a horrific dev environment, if randomly shut down with 30 seconds notice.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#26

Post by jhugo »

cma wrote: post #25
shock-value wrote: post #6

Earlier quoted context omitted.

It's probably a niche use case, but there is some utility to having a guaranteed daily shutoff. For example, you might spin up an instance as an on-demand remote dev environment, and the 24 hour cutoff ensures it doesn't accidentally get left on (over a weekend, for example). This would be easy to work around, but nonetheless could lead to unexpectedly high charges if you were relying on this behavior only to have it…

Tha sounds like a horrific dev environment, if randomly shut down with 30 seconds notice.

I can imagine it being useful if you needed to test something out for an hour or two and were worried you might forget to shut it down. Your charges are capped at 24h no matter how badly you screw up.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#27

Post by [deleted] »

roca wrote: post #20
snewman wrote: post #13

Earlier quoted context omitted.

Except that (presumably) there is no guarantee that any spot instances are available. You could wind up in a state where you have no instances running. No user-facing production service can tolerate that.

Just spin up regular VMs if you can't get spot instances when you need them.

[deleted]

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#28

Post by cma »

forty wrote: post #15
zamalek wrote: post #12

Earlier quoted context omitted.

If your code can survive chaos monkey, it can trivially weather this. Netflix, as one example, could run their entire infrastructure on these VMs.

Well except that on peak hours their infrastructure would disappear :) You always need some traditional instances to serve things that need to be available at all time, like your website. Now you could run things using spot instances and switch to traditional during peak hours but that requires confidence that enough capacity is going to be available when you need it (which is going to be at the same time as everyone…

Apparently some just gamble that by having some instances of every time, they won't all go down:

"However, your cloud provider can pull these virtual machines out from under you at any time without notice (because someone else is willing to pay more). So how do you lower the probability of losing all of your instances to nearly zero? Cloud providers generally need to have a large buffer of capacity and they have many different virtual machine configurations with different CPU and memory sizes. So, don’t just spin up virtual machines of a single configuration type, spin up lots of them from many different configuration types!"

https://blog.comma.ai/scaling-for-10x-user-growth/

I'm not sure if they still have a fallback of traditional instances.

Re: GCP releases Spot VMs, the next generation of Pre-emptible VMs

#29

Post by andoma »

travisd wrote: post #3

I don’t see this in the article, but is there any reason why they’d introduce a new VM type rather than just removing the 24hour limit of preemptible VMs?

I run a small CI system which starts relatively beefy preemptible instances for jobs. Normally those instances are terminated by the job scheduler as soon as there's nothing more to do. But it's a great peace of mind knowing they can't run for more than 24h if my job scheduler screws something up for whatever reason. I see it as a feature rather an a limitation for my use case.
Post reply on HN