121–130 of 149 posts
I really wish AWS would allow users to cap billing. Something that freezes all AWS services if the monthly bill exceeds X would make me a lot more comfortable when experimenting with AWS.
I really wish AWS would allow users to cap billing. Something that freezes all AWS services if the monthly bill exceeds X would make me a lot more comfortable when experimenting with AWS.
Have you looked into a tool like Gorilla Stack - https://www.gorillastack.com/ ? (Note: I have no investment in the company)
Without wishing to be too promotional, you can set a trigger to shut off EC2 when a cost threshold is reached. You can currently automate shut off of RDS but not yet from a cost threshold (that will be available shortly).
TLDR: we can do a lot of what you ask but not all of it.
Feel free to reach out if you'd like more info.
I really wish AWS would allow users to cap billing. Something that freezes all AWS services if the monthly bill exceeds X would make me a lot more comfortable when experimenting with AWS.
Earlier quoted context omitted.
You're right, this is effectively a serverless mode. But the serverless instances that are currently available (at least on aws) aren't powerful enough for some applications. For those of us stuck in the middle, needing big machines with serverless behaviour, this is a massive win.
I'm in the same boat as you: data science workloads that run intermittently on big hardware. What are you running your big jobs on? Because I'm currently using Batch, but given you've got to wait for the compute environment/VM to start up (if it's not already running), and that's a pain because it takes forever to startup. I wish I could just run containers on large hardware the same way we can run lambda's: press th…
The way we have it set up is:
- simple job queue with RQ (redis)
- monitoring watches the queue and pumps a metric into Cloud Watch (there are a few different types of job and it calculates a single aggregate value for "queue pressure")
- autoscale then sets the desired capacity for a fleet of r4.2xlarge machines (somewhere between 1 and 20)
- the autoscale config protects all those machines from scale-in so they have to be shutdown externally
- each of those machines has a cron on boot that tracks the start time
- this enables a cron to run just before the end of each hour. If that machine isn't doing anything at the time, it will shut itself down
- the machines are set to terminate on shutdown so they die completely
- additionally, we've hacked RQ so that workers that are closer to death will move themselves to the back of the queue more frequently. This ensures that we have a higher chance of not being busy / shutting them down at the end of the hour.
Earlier quoted context omitted.
I'd be surprised if it'd be that much additional work, if the local SSD options are NVMe -- which is also PCI-e like GPU's.
Wait what, would they expose the raw SSD pci device to your vm? What's stopping you from scraping all the leftover data from the previous customer?
I really wish AWS would allow users to cap billing. Something that freezes all AWS services if the monthly bill exceeds X would make me a lot more comfortable when experimenting with AWS.
I lost $1200 to an s3 mistake when I was broke years ago, it really sucked.
Earlier quoted context omitted.
I lost $1200 to an s3 mistake when I was broke years ago, it really sucked.
I mistakenly got a 1400 dollar bill . I just called and said it was a mistake and I don't want to pay it and they said ok. Probably too late for you but maybe helpful for someone in the future.
Earlier quoted context omitted.
They would legitimately lose a lot of money if they did this now (and I would argue, wouldn't make up for it in market share). Most companies I work with who switch to AWS think it's a 1:1 conversion in terms of cost from a data center and pretty much just "leave AWS on" not realizing that they are not only paying for computing cost but the hidden cost of being able to scale up more instances quickly.
It's really really difficult to be able to turn on and off instances as needed, like turn a test instance on in the morning when developers come in and turn it off in the evening when they leave.
Earlier quoted context omitted.
The reserved instance pricing model is really poorly implemented. You have to commit to an instance class and a region, and if you need to change you can, but there is some secondary market. When compared to a sustained use model, it is a disaster. Managing all that in a large org is a real pain in the ass.
There are so many options for RI purchases honestly. Standard vs. Convertible: Convertible allows you to switch between instance families (like c3, m3, m4, i2, r3 etc...) but it requires you to make a 3yr commitment, commit to a specific AZ and doesn't offer the same level of savings you'd get with a 3yr standard, I think it's closer to the savings you'd get with a 1yr Standard RI. Standard RI's come in 1yr or 3yr co…
I really wish AWS would allow users to cap billing. Something that freezes all AWS services if the monthly bill exceeds X would make me a lot more comfortable when experimenting with AWS.