Live data from Hacker News

Setup AWS Cloudwatch Monitoring and Alerts Using Bash Scripts

themythicalengineer.com

11–20 of 34 posts

Re: Setup AWS Cloudwatch Monitoring and Alerts Using Bash Scripts

#11
post #8

Earlier quoted context omitted.

If you don't like Terraform then you should use CloudFormation, not bash scripts.

Oh no that’s even worse.

If you prefer imperative infrastructure creation to declarative then I think you're doing something wrong. Both Terraform and CloudFormation are quite easy to manage compared to writing and managing scripts (bash or otherwise).

Re: Setup AWS Cloudwatch Monitoring and Alerts Using Bash Scripts

#12
post #8

Earlier quoted context omitted.

Oh no that’s even worse.

If you prefer imperative infrastructure creation to declarative then I think you're doing something wrong. Both Terraform and CloudFormation are quite easy to manage compared to writing and managing scripts (bash or otherwise).

I’m only having a gripe. I use terraform because it’s the least bad tool, not because it’s the best. I wish for better.

Re: Setup AWS Cloudwatch Monitoring and Alerts Using Bash Scripts

#14
post #2

Not sure why you’d ever do this instead of using terraform.

Yes.

Using terraform for this is great is because it removes the unwanted alarms.

I had to create alarms when the instances auto scale and wrote a python script using cdktf and now the Jenkins job handles it. It even updates the cloudwatch dashboard.

Re: Setup AWS Cloudwatch Monitoring and Alerts Using Bash Scripts

#15
We've been looking at making CloudWatch (CW) alarms an automated part of our infra. Here are some findings that may help:

- The semantics of CW seem convoluted. But once you stare at API docs for long enough, the core concepts are easy to grok: Metrics (regularly submitted from machine to CW), Alarms (abstractions for defining the logic of an alarm based on behavior of Metrics), and SNS Topics (could be just an email address, for what to do when an Alarm goes off).

- Once you get the data model right, all implementations (click ops, terraform, bash via awscli, boto3, etc) are all visibly identical.

- Some Metrics come for free, e.g. CPU usage is reported by any EC2 instance to CW. For some other Metrics, notably disk and memory usage, you need to configure your instance to report them to CW. This is where the OP's monitoring scripts come in.

- The monitoring scripts and the cron config the OP refers to are deprecated [0]. Instead there's a new CloudWatch Agent [1]: you install the package on your EC2 instances, provide a configuration file to it, and you're set.

[0] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scri...

[1] https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...

Re: Setup AWS Cloudwatch Monitoring and Alerts Using Bash Scripts

#16
post #15

We've been looking at making CloudWatch (CW) alarms an automated part of our infra. Here are some findings that may help: - The semantics of CW seem convoluted. But once you stare at API docs for long enough, the core concepts are easy to grok: Metrics (regularly submitted from machine to CW), Alarms (abstractions for defining the logic of an alarm based on behavior of Metrics), and SNS Topics (could be just an email…

You can install the CWAgent on any host, they don't have to be EC2 instances:

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...

Re: Setup AWS Cloudwatch Monitoring and Alerts Using Bash Scripts

#19
post #7
post #5

Earlier quoted context omitted.

Agreed, I swapped my team from teraform to Ansible to SAM... SAM has been the most reliable and resilient and stable for my use cases (general serverless)

SAM is cloudformation. cloudformation is the thing to use if you’re on the AWS cloud

CloudFormation is without a doubt the worst cloud technology I have ever used.

Re: Setup AWS Cloudwatch Monitoring and Alerts Using Bash Scripts

#20
post #3
post #2

Not sure why you’d ever do this instead of using terraform.

As someone who spends two hours a day dealing with buggered terraform state and upgrading terraform and dealing with terraform bugs I can see it. It’s one of those things that really works pretty well but there are enough edge cases to make it slightly soul sucking.

This sounds like a lack of understanding of terraform. We use Terraform pretty heavily and I've rarely seen bad states across our whole org, and the few that I do see are usually people who don't know the core concepts (often non-devops engineers).

Terraform has its faults, but it is the best in its class, especially when you need to manage infrastructure beyond a single cloud provider (e.g. we manage our datadog monitors and dashboard, pagerduty alerts and much more). The only other thing that would probably thrash it is pulumi, which has similar concepts, except you can many different languages as opposed to HCL (no CDK doesn't count because it is very immature still and last I checked it only supported one or two languages).

Post reply on HN