Ah, finally. They've ruined my idea for an optimal EMR job runner. Under the old system, if you have a linearly scalable Hadoop job, it's cheaper to, say, use 60 instances to do some work in an hour vs 50 instances to do the work in 70 minutes, assuming you're getting rid of the cluster once you're done. No more!
AWS announces per-second billing for EC2 instances
91–100 of 149 posts
Re: AWS announces per-second billing for EC2 instances
#92Earlier quoted context omitted.
Wait what, would they expose the raw SSD pci device to your vm? What's stopping you from scraping all the leftover data from the previous customer?
Probably something along the lines of secure erase. Most modern SSDs/NVMe drives are encrypted by default in firmware. All the firmware needs to do is throw away the old keys and generate new ones. It's better than zeroing the drive as there is no wear to the write cycles and guarantees that the slack space in the SSD is also cleared, which DD'ing to /dev/nvme0 wont be certain of. The nvme-format tool can be used for…
On newer SSDs, the sanitize command would be preferable for this use over the format command. IIRC, the format command doesn't require quite as strong a security guarantee as the sanitize command: the latter ensures that user data is cleaned from both the flash and all buffers, CMBs, etc.
Re: AWS announces per-second billing for EC2 instances
#93Earlier quoted context omitted.
Tag it with something like shutdown=non-biz-hours, or can even specify a schedule that a lambda function can pick up each minute or hour and parse to stop/start any instance that falls within that window.
Well that works if you're a startup in one timezone with no teams depending on your service. If you work on a larger company, it's quite possible the sun never sets on your project (west coast, east coast, Europe, India, Australia or SE Asia)
If the sun never sets on your project, why would you want to shut the instance down?
Re: AWS announces per-second billing for EC2 instances
#94Earlier quoted context omitted.
Well that works if you're a startup in one timezone with no teams depending on your service. If you work on a larger company, it's quite possible the sun never sets on your project (west coast, east coast, Europe, India, Australia or SE Asia)
If the sun never sets on your project, why would you want to shut the instance down?
One reason would be that the savings is worth the increased latency. Depends on the situation.
Re: AWS announces per-second billing for EC2 instances
#95I once considered writing an EC2 autoscaler that knew the exact timestamps of the instances so that it could avoid shutting down VMs that still had 59 minutes of "free" time left because they'd been up across another hour-long threshold. That sort of nonsense logic shouldn't be useful, but Amazon was giving a huge economic incentive for it. This is certainly a long time coming.
Remember that AWS has been in the game for over a decade. Per-hour billing was amazing when it came in. Also, is the economic incentive really that huge? Or is it just a nicety?
it's totally dependant on your workload. for some users there will be absolutely no difference, for others it could easily be thousands or tens of thousands of dollars of savings over a year.
Re: AWS announces per-second billing for EC2 instances
#96Earlier quoted context omitted.
It's really really difficult to be able to turn on and off instances as needed, like turn a test instance on in the morning when developers come in and turn it off in the evening when they leave.
You can implement this with a simple auto-scaling group or write a small Lambda function which runs at a certain time to start/stop the instance
You typically want exactly 0 or 1 instances for testing instances. I don't recall autoscaling groups able to do that.
Re: AWS announces per-second billing for EC2 instances
#97Earlier quoted context omitted.
They would legitimately lose a lot of money if they did this now (and I would argue, wouldn't make up for it in market share). Most companies I work with who switch to AWS think it's a 1:1 conversion in terms of cost from a data center and pretty much just "leave AWS on" not realizing that they are not only paying for computing cost but the hidden cost of being able to scale up more instances quickly.
It's really really difficult to be able to turn on and off instances as needed, like turn a test instance on in the morning when developers come in and turn it off in the evening when they leave.
Re: AWS announces per-second billing for EC2 instances
#98Earlier quoted context omitted.
If you're concerned about cost, AWS is almost never the right place to host to begin with.
You miss the point. The lifetime of a web request, for example, can be measured in milliseconds. It is now possible, technically anyway, for operating systems to boot, service the request and disappear. There needs to be pricing models that reflect computing models like this.
Isn't that what lambda is all about? Sub-second billing?
Re: AWS announces per-second billing for EC2 instances
#99Earlier quoted context omitted.
There are a wide range of tiny operating systems that can boot in a matter of milliseconds. The applications are "whatever you want imagine" but yes one application is building FAAS Function As A Service in which the operating system carries out a single function. Put anther way, Docker is complex, overweight, and requires re-implementation of much computing infrastructure. You can meet many of the same goals as Dock…
I'm somewhat amused by the idea of booting an operating system from scratch to service a single request being described as "much more simple" than the alternative of, y'know, having a single instance serve many requests.
[deleted]
Re: AWS announces per-second billing for EC2 instances
#100Earlier quoted context omitted.
You can implement this with a simple auto-scaling group or write a small Lambda function which runs at a certain time to start/stop the instance
You typically want exactly 0 or 1 instances for testing instances. I don't recall autoscaling groups able to do that.
I have an autoscaling group that goes from 0 to 1 every day for some batch processing. It took a bit of finagling but it's definitely possible.