Cloud computing made financial control the new engineering. Unless your service is running at an astronomical scale, nobody cares that your refactor saved 50 MB of RAM. The cloud vendors don't care if your workload needs 128 MB RAM, 256 MB RAM, 8 GB RAM. They'll run anything. Just pay for it. The industry is in a weird spot right now where Finance doesn't understand Engineering well enough to provide proper direction…
I always find it so weird that the first time in my whole career that I ever got authority to spend without literally any approval was when we got our cloud account. Every single other thing, even a $10 cab ride, had to be authorised. But when we signed up for AWS I just do whatever I do and finance pays the bill. I still do not understand how this happened.
81% of IT teams directed to reduce or halt cloud spending
161–170 of 235 posts
Re: 81% of IT teams directed to reduce or halt cloud spending
#162Earlier quoted context omitted.
I think growing food is the wrong analogy, and cooking is better. Growing food is more like designing and manufacturing servers. Cooking is more like taking these goods and putting them to use. There are plenty of places that host their own servers, including people that have these servers in their home or office. But far fewer produce these machines. You could also say that cooking food is dumb. What's wrong with ha…
The major cloud providers are designing their own servers, so we’re back at “growing your own food”.
Facebook does not use cloud providers and designs their own servers.
The very large compute uses and cloud providers have incentives to grow their own food. Doing so can save them a few percent at scale. That savings is both not passed on to you and even if it was is not significant at small to medium scales.
Re: 81% of IT teams directed to reduce or halt cloud spending
#163Instead, I find myself being slowed down greatly: "Having a log shipping solution would be nice here, but I cannot set it up, because there's not enough RAM. Static code analysis? Not enough RAM. Another back end API instance for that new front end we're working as a replacement for the old one? Not enough RAM. Wanting to test how well scheduled processes run overnight with larger amounts of data? Too few CPU cores. Want more container images stored in your own Nexus instance? Not enough disk space. Want another database instance for CI or testing destructive migrations? Not enough of anything."
Which is the problem that the cloud largely solves, at the expense of way much money spent than necessary. But hey, when you have to deal with red tape to get more resources on-prem, that's just a way of managing the overall process and manages risks that you might not care about in the moment when you "just want to get things done"; albeit that's not to share that you shouldn't care about them.
So there are definitely problems with either approach (even cloud setups might have to go through someone and Jira/Redmine issues, as opposed to self-service), albeit at the end of the day most of those feel social/economical - after all, those large corps need their cloud service profit margins.
Re: 81% of IT teams directed to reduce or halt cloud spending
#164I have access to a button that I can trivially press, that a coworker recently told me generates a $50 AWS bill.
There's various levels of foot-gun clicks that can generate AWS bills many orders of magnitude larger. Reserved Instances are a really quick and easy way to blow a few million with one click. Europe (Frankfurt), 3 year, all-upfront, Windows w/ SQL Enterprise, Dedicated Tenancy, u-12tb1.112xlarge instance at $5,926,378 up-front. The discounted $225/hour rate for it disappears into the background noise. (No, I'm not wi…
Re: 81% of IT teams directed to reduce or halt cloud spending
#165How many would say this normally? I.e., what's the baseline? Meanwhile, how many are giving their teams the resources (i.e., people) necessary to do the analytics on the bill required to find & cut costs? Every cloud I've worked with has been terrible at billing in a way that could be reasonable investigated. At one point we literally pulled all the billing data into a SQLite database. (And that was a great idea, as…
> where the money is going AWS in particular is handsomely engineered so you can't answer this sort of question, as well as making people spend more without need. - Per resource cost is turned off by default. Turning it on costs money. How much? ¯\_(ツ)_/¯ - Services like S3 will bill you per API call. How do you know who is calling your APIs? Using Cloud trail, which is turned off by default. Turning it on for this s…
Re: 81% of IT teams directed to reduce or halt cloud spending
#166Re: 81% of IT teams directed to reduce or halt cloud spending
#167How many would say this normally? I.e., what's the baseline? Meanwhile, how many are giving their teams the resources (i.e., people) necessary to do the analytics on the bill required to find & cut costs? Every cloud I've worked with has been terrible at billing in a way that could be reasonable investigated. At one point we literally pulled all the billing data into a SQLite database. (And that was a great idea, as…
But how does it take a over week to answer the question? Sounds like the more basic problem is that we keep ending up working in companies where the "higher ups" are constantly shoving decisions down our throats. Which, ultimately, is probably the root cause of most of the caginess, the yak shaving, the compunction to use cloud databases and cloud-everything, and the silly support tickets that stay open forever and e…
If you don't fund, hire, and staff your support org with good people at multiple tiers, don't be surprised when you get crap. There just aren't enough good minds to service the incoming workload.
Oh, and for god's sake have a decent "to engineering" escalation path for the support folks.
Unfortunately, the support function is almost always a cost center and beholden to only, if anything, response time related SLAs. Instead of customer satisfaction, etc.
Re: 81% of IT teams directed to reduce or halt cloud spending
#168Earlier quoted context omitted.
Is this the same for child accounts created via AWS Orgs?
I guess so, unless we were severely misinformed. We had historically one big AWS account, we are now splitting it into more fine-controlled accounts and migrated under a "root" account following AWS best-practices to have unified billing and we can enable the Business support only on the big (legacy) account where we run 100% prod and other things. I don't think we had the option to enable it on the root account and…
Re: 81% of IT teams directed to reduce or halt cloud spending
#169Earlier quoted context omitted.
So there are three choices, essentially to argue over: - run on demand instances (maybe mix in spot) - run fargate instances (maybe mix in spot) - reserve them. Every company hates #3, but it's usually the best choice if you can't go spot - you can swap them within the family for size, use them for baseline, it's fine. You're incorporated right? So if you fold, it's just another debt on the pile, stop worrying. Go no…
IMHO Reservations in AWS are good if you have a US Bank Account. Why? Because there is a secondary market (run by AWS) where you can buy and sell reservations. If you have a 1-year project that will have a 1-year extension, then get a 3-year reservation (with some amount upfront) that you can sell. Your second best option is a Savings Plan.
Re: 81% of IT teams directed to reduce or halt cloud spending
#170Earlier quoted context omitted.
>The solution never tried: share the spoils of efficiency with your teams. Funny enough, for all the problems with studying business administration, they do teach this as part of the principal-agent problem. If the principal wants something done, and the agent has no incentive to do it, you need to give the agent some of the incentive so their interests are aligned with yours. At the same time it's something you rare…
Rather they than just "they don't like to share", the BA types don't do this because in one of their other classes they have been taught about the downsides of such incentives, like: - Corrosive effects on team cohesion, when separate ICs compete to see who can find cost savings first. - The "cobra effect", where engineers will deliberately introduce cost inefficiencies so that they can fix their own bugs later. - (R…
Commissions, metrics, and goals and heavily used by sales across many companies. I’m sure there is some abuse but it is still heavily used.