Show HN: Infracost (YC W21): Be proactive with your cloud costs

#1

Post by hkh » Wed, Aug 09, 2023, 1:01 PM UTC

Hi, we are Ali, Hassan, and Alistair, co-founders of Infracost (https://www.infracost.io/). Infracost helps engineers see the cost of each Terraform change before launching resources. When changes are made, it posts a comment with the cloud cost impact. For example, “you’ve added 2 instances and volumes, and change an instance type from medium to large, your bill will increase by 25% next month, from $1000 to $1250 per month”.

We launched in February 2021 (https://news.ycombinator.com/item?id=26064588), and Infracost is now being actively used by over 3,000 companies. However, there is a shift happening in the cloud cost management space. New teams, called FinOps teams (a combination of "Finance" and "DevOps"), are being formed within companies to manage cloud costs.

One of the first tasks assigned to these teams is to determine "who is using what" - that is, which teams, business units, products, etc. are spending the most on cloud. To accomplish this, they use tags. Tags are labels that all cloud resources should have and are key-value pairs. For example, a server could be tagged with: product=HackerNews; environment=production; team=blueTeam. So if resources are not tagged properly, then you can’t tell who is using what.

However, FinOps teams face challenges because their tools are reactive. These tools begin by analyzing cloud bills and providing visibility of tags from there. This means that they are looking at resources that are already running in production and costing money. A customer recently shared, “I want all resources to be properly tagged. But if they are not, I would rather a resource not be tagged at all than be tagged incorrectly.”

My "aha" moment! FinOps teams can define a tagging policy that can be validated in CI/CD before resources are launched. This is important because if code is shipped with the wrong tags, FinOps teams will have to fight for sprint time to fix them. Even if you shut down an untagged resource directly in the cloud, the next time Terraform runs, the resource will launch again with no tag. You need to fix the issue at its root.

I’d love your feedback on our solution to the tagging problem. You define your tag key-value policy in our SaaS product, and Infracost checks all Terraform resources per change. If anything fails the policy, it posts a comment with the details of which resources need tags, and what the allowed values are. Once fixed, it will let the code be shipped to production.

Try it out by going to https://dashboard.infracost.io/, setting up with the GitHub app or GitLab app, and defining your tagging policy. It will then scan your repository and inform you of any missing tags and their file and line number. You can use the free trial, but if you need more time, please message me and I’ll extend it for you.

I would also love to hear how others ensure that the correct tag keys and values are applied to all resources, and whether this is done proactively or reactively. Additionally, I would be interested in hearing about any lessons learned in the process.

Cheers

Re: Show HN: Infracost (YC W21): Be proactive with your cloud costs

#2

Post by plasma » Wed, Aug 09, 2023, 1:49 PM UTC

I’m not a target user, but you mentioned the tagging problem and git integration, perhaps you could infer at least the git user responsible for each resource cost (git blame the TF file and identify the username who added the resource) as a minimum amount of detail provided out of the box?

Re: Show HN: Infracost (YC W21): Be proactive with your cloud costs

#3

Post by keepamovin » Wed, Aug 09, 2023, 1:51 PM UTC

I love it! But as human processes go, it will need to surmount the "flaky tests" problem of, "let's just turn off this test because it's flaky and we need to merge this branch". I guess that means FinOps teams will still have to fight to be heard, but I think you are helping shift a lot of their burden!

What remains seems more like organizational dynamics, but what are your thoughts?

Re: Show HN: Infracost (YC W21): Be proactive with your cloud costs

#4

Post by superdeeda » Wed, Aug 09, 2023, 1:56 PM UTC

Sounds useful!

We're using service control policies to enforce tagging on certain resource types, and retroactively for the rest.

Considering to use a "shift-left" tool as well, but it would need to support Terraform, CDK, Serverless and Cloudformation.

Re: Show HN: Infracost (YC W21): Be proactive with your cloud costs

#5

Post by akh » Wed, Aug 09, 2023, 2:06 PM UTC

keepamovin wrote: post #3

I love it! But as human processes go, it will need to surmount the "flaky tests" problem of, "let's just turn off this test because it's flaky and we need to merge this branch". I guess that means FinOps teams will still have to fight to be heard, but I think you are helping shift a lot of their burden! What remains seems more like organizational dynamics, but what are your thoughts?

Great point - indeed FinOps teams consistently rank "empowering engineers to take action" as their number 1 challenge (https://data.finops.org) - and by that they mean the human and organization dynamics of the culture change they want to create across the org.

The testing analogy is a good one as this feature also shows the engineers the current "failing policies" on the main branch too, so whilst they could merge the pull request without fixing the tagging issue, it'll just get added to the list. And maybe like tests, they group them into one task and go through to fix them all every so often to get the main branch back to green!

Re: Show HN: Infracost (YC W21): Be proactive with your cloud costs

#6

Post by beaviskhan » Wed, Aug 09, 2023, 2:13 PM UTC

I have used this tool in the past, though free tier only. It was easy to get up and running and easy to plug into a CICD pipeline. The problem we had with it in practice was that we largely preferred serverless technologies in AWS where the cost depended mostly or even completely on actual usage - things like Lambda invocations, SQS operations, or autoscaling ECS services, for example. In this case the estimates we got from Infracost were not very useful. Providing a meaningful cost estimate requires projecting usage, which is something that our development teams were very bad at, if they could be bothered to care at all.

I like the idea of implementing tagging enforcement in the pipeline. In a perfect world you would use cloud policies to do this, but in practice this is a big loser in AWS where a staggering number of resources are created by one API call and then tagged as a followup API call, meaning an SCP to prevent launch of untagged resources won't ever work.

Re: Show HN: Infracost (YC W21): Be proactive with your cloud costs

#7

Post by akh » Wed, Aug 09, 2023, 2:17 PM UTC

plasma wrote: post #2

I’m not a target user, but you mentioned the tagging problem and git integration, perhaps you could infer at least the git user responsible for each resource cost (git blame the TF file and identify the username who added the resource) as a minimum amount of detail provided out of the box?

Interesting idea! The pull request authors are shown out of the box but we hadn't thought of using git to find the user for each resource on the main branch. Most organizations end-up tagging the resources with some sort of owner or team so they can group the costs using that and track it per team/service/product over time. That's often how FinOps teams start to create a sense of ownership for cloud costs amongst teams.

Re: Show HN: Infracost (YC W21): Be proactive with your cloud costs

#8

Post by aliscott » Wed, Aug 09, 2023, 2:18 PM UTC

superdeeda wrote: post #4

Sounds useful! We're using service control policies to enforce tagging on certain resource types, and retroactively for the rest. Considering to use a "shift-left" tool as well, but it would need to support Terraform, CDK, Serverless and Cloudformation.

Awesome, yeah we’ve seen people using this method and the main complaint we’ve heard is this is annoying for developers since it blocks their deployments when they run `terraform apply`, so they need to create new pull requests and wait for another code review. Combining both can definitely help with this.

Re: Show HN: Infracost (YC W21): Be proactive with your cloud costs

#9

Post by cube2222 » Wed, Aug 09, 2023, 2:22 PM UTC

superdeeda wrote: post #4

Sounds useful! We're using service control policies to enforce tagging on certain resource types, and retroactively for the rest. Considering to use a "shift-left" tool as well, but it would need to support Terraform, CDK, Serverless and Cloudformation.

Not sure if with shift-left you mean specifically shifting left infracost and FinOps or general Infrastructure-as-Code shift left.

In case it's the latter, I can recommend Spacelift[0] - a specialized CI/CD tool for IaC and supports all the tools you've mentioned. It basically helps you build policies and orchestrate your infra (don't want to go into too much detail in this comment) to scale it to bigger teams and setups. Policies to enforce tagging would indeed be a good example.

It integrates with infracost too, but obviously just for the tools infracost works with, no CloudFormation.

Disclaimer: Work at Spacelift so obviously take the recommendation with a grain of salt, but I do legitimately think it's a great tool.

[0]: https://spacelift.io

P.S. Congrats on the Show HN Infracost team!

Re: Show HN: Infracost (YC W21): Be proactive with your cloud costs

#10

Post by alexambarch » Wed, Aug 09, 2023, 2:27 PM UTC

I've seen Infracost around and think it looks cool, do you have any plans to add support for Pulumi? One advantage Terraform seems to have over Pulumi is the ecosystem of tools that support it.