31–40 of 78 posts
IE, you will save a bundle if you are currently buying 3k provisioned IOPS vs just paying for 1 TB of storage.
More here: http://blog.textit.in/why-buying-provisioned-iops-on-rds-may...
"Run an entire tech company in the cloud, or run only a single [big] project requiring more than 10 servers? Google Compute Engine
Run less than 10 servers, for as little cost as possible? Digital Ocean
Run only beefy servers ( > 100GB RAM) or have special hardware requirements? IBM SoftLayer or OVH"
https://thehftguy.com/2016/06/08/choosing-a-cloud-provider-a...
https://thehftguy.com/2016/11/18/google-cloud-is-50-cheaper-...
Serverless / Zappa, hosting my API’s for 10x decrease in monthly costs. https://github.com/Miserlou/Zappa
We wrote about how we analyzed our AWS usage https://segment.com/blog/spotting-a-million-dollars-in-your-... , and then some of optimizations we made to cut down costs https://segment.com/blog/the-million-dollar-eng-problem/ .
We stopped storing our IoT raw data in databases. We still need to search it, but now we store only metadata in the database (we know what we will search by, so we can make appropriate metadata) and store the raw data in S3. So any searches are in the DB, using the DB for what it is good at. This means that our storage / database cost approaches just the S3 cost, because our metadata is ~0.001 the size of the raw dat…
We cut our spot expenses by 20% , which were already cheaper than on-demand by about 70%.
No matter how we played with the price calculator, Google was twice as expensive than our spot fleet algorithm
I’m working on https://www.dormantbear.com which is an aws scheduler to turn instances on and off for certain times of the day.
I use it for my own VPN box which I only require during office hours.
The main commercial use case is for turning off staging environments when not needed during out of office hours. For setups that are fixed and don’t spin up test environments on the fly.
For security you should provide aws creds that can access your staging boxes with permission to list instances and turn them on and off. Even though they are symetrically encrypted.
The service is free as I have not written billing or marketing pages yet.
We stopped storing our IoT raw data in databases. We still need to search it, but now we store only metadata in the database (we know what we will search by, so we can make appropriate metadata) and store the raw data in S3. So any searches are in the DB, using the DB for what it is good at. This means that our storage / database cost approaches just the S3 cost, because our metadata is ~0.001 the size of the raw dat…
How slow is it to retrieve the data from s3 though?
Let's be specific on speed. Most of the time, the MB or two needed for a plot is a fraction of a second, which our customers can deal with. That retrieval is a single object from S3, the way we organize things.
Having said that, the talk I linked to has some great advice - use the "folder" structure to write data so you don't search, you just use the naming scheme to do a direct object read. In addition, we can keep most reads to a single object, which is fairly fast.
As is always true, you will need to test to see if it fits your speed needs. But even with just the naming scheme and meta data in the database to eliminate all searches on S3, the speed works for us.