121–129 of 129 posts
Earlier quoted context omitted.
They already replicated the S3 API as an alternative to the Google Cloud Storage API: https://cloud.google.com/storage/docs/migrating#migration-si...
I get the impression that's pretty common, Cloudflare's new product is S3 compatible too.
Earlier quoted context omitted.
Yeah, GCP spot instances don't vary in price like AWS ("The price adjustments will occur at most once a month").
There is no longer bidding for spot prices on AWS. They moved to model where pay the fixed spot price which isn't based on capacity. https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pri...
Earlier quoted context omitted.
I run a small CI system which starts relatively beefy preemptible instances for jobs. Normally those instances are terminated by the job scheduler as soon as there's nothing more to do. But it's a great peace of mind knowing they can't run for more than 24h if my job scheduler screws something up for whatever reason. I see it as a feature rather an a limitation for my use case.
Yes, but they could just provide a flag `maxLifetime` which defaults to `24h` but can be set to 0. (It can even only accept those two values) Instead they created a whole new API for basically the same feature.
Earlier quoted context omitted.
All our pernos.co infrastructure uses AWS spot VMs. Batch processing runs on spot and fails over to regular VMs as necessary. CI jobs run on spot. Our front-end server running the Web site and our application also runs on spot and automatically fails over to a regular VM if it gets terminated. Termination is rare.
How do you coordinate the failover? Custom scripts or?
Earlier quoted context omitted.
Just spin up regular VMs if you can't get spot instances when you need them.
Lack of spot VM means that all spare capacity has been taken up by the regular VMs, meaning that it spots are gone, so are regular ones.
Earlier quoted context omitted.
In AWS we just `sudo shutdown +1440`. Then we can cancel the shutdown later if we need to.
In AWS I set up a few lambdas to shut down my instances automatically because I've had some cases where everything crashes so thoroughly that the shutdown doesn't go off.
Earlier quoted context omitted.
There are tools that enable this (e.g. look at temporal.io). As for external storage: yes, you do generate more metadata, but that's often a tiny fraction of the cost saved by moving to spot machines. In general, these tools are adopted more for increasing reliability of existing systems, but I predict they would be a neat fit to run them on spot machines.
+1 for Temporal. Using in prod, its great
Earlier quoted context omitted.
Tha sounds like a horrific dev environment, if randomly shut down with 30 seconds notice.
"Horrific" is massive hyperbole, at least for my use case. (I actually use a GCP preemptible instance in this manner for a personal project, and spot instances will have the same shutoff risk.) I edit application code on my (relatively underpowered) laptop, and automatically mirror it to the instance where the running services picks up any changes, and recompiles and relaunches as needed. It's a fairly chunky app cod…