I attended aws workshop, several years ago, there I realized, that this vast ecosystem of infrastructure services require a popularisation effort of similar scale. And not just in plain English. IAM that days was not very clearly understandable, search supported only ascii and was not really documented, now that ecosystem is in order of magnitude larger and more complex. And efforts like cloudonaut's should be greatly appreciated. For the greater public good. Thank you, man!
AWS mistakes to avoid
51–60 of 276 posts
Re: AWS mistakes to avoid
#52The biggest mistake I've seen with AWS (and committed myself), is not reading the manuals for the services you're using. While some people complain about the AWS manuals not being complete, there is still a lot of good information in there that you might miss if you're just clicking through the console.
Re: AWS mistakes to avoid
#53I like CloudFormation. Unfortunately it is very unwieldy to write CloudFormation templates directly, and we're not about to start using the AWS CFN GUI editor! It seems like the assembly of the AWS ecosystem. Does anyone else have a favourite hammer for this particular nail? I'd love to have something better than our home-baked solution, but I'm yet to find anything which doesn't introduce other flaws, such as an inc…
If you'd consider something other than CloudFormation, there is also Hashicorp's Terraform. It has an AWS provider ( https://terraform.io/docs/providers/aws/index.html ) which creates resources and maintains the state in a file that you can store in version control ( https://terraform.io/docs/state/index.html ).
I second the Terraform suggestion...my team loves it. But we've found storing state in version control to be clunky. Storing state remotely in Consul has been less problematic for us, though S3 would also work for those that don't have a running Consul cluster.
What I love most about Terraform is that we can include the output of terraform plan in pull requests that make infrastructure changes. Then our continuous deployment process runs plan again and requires an identical output before running apply. This both makes it easier for team members to review changes but also ensures that we don't accidentally destroy infrastructure, which is really easy to do with a lot of these infrastructure-as-code tools.
The other thing that Terraform has going for it over CloudFormation is for hybrid cloud deployments, since it can provision infrastructure in vSphere and OpenStack as well as AWS.
Re: AWS mistakes to avoid
#54I'll add one that was especially common for people coming off the year of free tier a couple years ago. I'm not sure that AWS has changed it yet. 6. Not starting a box/instance/database and forgetting it's running until you receive the bill after your free tier expires.
I expected to see something like this generalized in the article as
6. Not setting up billing alarms, http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/...
Re: AWS mistakes to avoid
#55I'd like to add, if using Elastic Beanstalk, don't directly attach an RDS instance when creating the environment. If you do, you won't be able to destroy your environment without also deleting the RDS instance. Instead, create the RDS instance separately, and just add the proper security group for the environment to be able to access the host. Then you can easily create a new eb environment with any config changes (there are some config changes you can't make to an eb environment without creating a new one from scratch) and then connect to your existing db.
Re: AWS mistakes to avoid
#56> There is no reason why you should manage your infrastructure manually. It's unprofessional! It's a mess!
Nonsense. Cloudformation has it's issues. It takes time to learn and implement. The templates can break, requiring the stack to be destroyed and remade. In the sample in the article, the database is in the same template as everything else - what fun that will be when an update breaks the template and you have to reapply the stack (which destroys the existing database).
Cloudformation is good, but it comes with caveats, and the idea that you should only manage an AWS stack with CF is utter tripe. As with everything, it depends on your use case.
Also weird is the article's demand of using autoscaling groups to monitor single instances. Why not just monitor them directly with cloudwatch?
> There is no reason - beside manually managed infrastructure - to not decrease the instance size (number of machines or c3.xlarge to c3.large) if you realize that your EC2 instances are underutilized.
This is wrong, too. Autoscaling takes time to scale up, and it scales up in stages. If you get sudden traffic, autoscaling can take too long. Again, it's about knowing your use case. Unfortunately for us, we can get sudden traffic when one of our clients does a media release and they don't tell us ahead of time, The five or so minutes it takes for instances to trigger the warning, start up a new set, and then attach these to a load balancer is too long for this particular use case, so we just have to run with a certain amount of excess capacity.
Autoscaling is awesome, but this article is way too didactic in it's No True Scotsman approach.
Re: AWS mistakes to avoid
#57You know what I've realized that's really important. More AWS tutorials is really needed. There's numerous of new programmers who want to learn AWS, but can't finish building anything because they get buried in documentation. I find there are a lot of high-level abstracted tutorials, but for the new services, there aren't a lot of detailed tutorials. For instance, an implemented cognito->gateway->lambda->dynamodb is…
> There's numerous of new programmers who want to learn AWS, but can't finish building anything because they get buried in documentation.
I agree partially. In the end the documentation is what you are going to need to read sooner or later. Or trainings equivalent to that documentation. Good tutorials are good for starting, but doesn't make you a professional.
I guess that in 10 years any one will be able to create websites for billions of internet connected users. But for now, as easy at it is, it is still complicated enough to require an expert. In the same fashion that 20 years ago you needed an expert to make a 3D game and nowadays there is plenty of technologies that allow you to do that with a limited amount of programming knowledge.
I have seen horrible things, usually security related, because non trained people think that they can achieve anything just with standard configurations and quick tutorials. And it looks like they where able to do it, until something really bad happens. Even people with long experience can make mistakes because it is a complex thing.
Re: AWS mistakes to avoid
#58My huge recommendation is to put production instances in a completely seperate region than development and staging instances. I actually just discovered that you can limit IAM API keys to a specific region, you just need to create a custom policy. The following policy is an example: { "Version": "2012-10-17", "Statement": [ { "Sid": "SOME-ID-HERE", "Effect": "Allow", "Action": [ "ec2:*" ], "Condition": { "StringEqual…
Alternatively, you can run two AWS accounts and get a single bill with 'combined billing'. One account being prod and one is test.
Re: AWS mistakes to avoid
#59I like CloudFormation. Unfortunately it is very unwieldy to write CloudFormation templates directly, and we're not about to start using the AWS CFN GUI editor! It seems like the assembly of the AWS ecosystem. Does anyone else have a favourite hammer for this particular nail? I'd love to have something better than our home-baked solution, but I'm yet to find anything which doesn't introduce other flaws, such as an inc…
If you'd consider something other than CloudFormation, there is also Hashicorp's Terraform. It has an AWS provider ( https://terraform.io/docs/providers/aws/index.html ) which creates resources and maintains the state in a file that you can store in version control ( https://terraform.io/docs/state/index.html ).
Strongly agree. Support for new AWS features hits Terraform much faster than CloudFormation (still waiting for CF support for AWS's managed ElasticSearch service that was unveiled two months ago--Terraform got it right away). Some of the critiques below are true... HCL is fine, but Terraform's interpolation syntax has a long way to go. That said, CF's JSON is way more painful to deal with. As for the other problems, they go back mainly to someone using a tool they don't fully understand. Yes you can get into some odd states in rare cases, but Terraform gives you the ability to rapidly build and tear down your infrastructure over and over if necessary to work out details and you have fine-grained control over which pieces are built how. Not only that but you can inspect the logs to see what's happening and if there are bugs, you can fix them yourself because the tool is open source and free. CloudFormation gives zero visibility, no fine-grained control, and it's completely opaque and where it's broken, you can't fix it.
Terraform is relatively new and improving rapidly. It has its problems, but it's light-years beyond CloudFormation. It's clear that Amazon doesn't place a high priority on making CloudFormation easy to use, or to support new features. The right approach to any problems with Terraform is not to spread FUD about it like below, but to contribute code fixes.
Re: AWS mistakes to avoid
#60You know what I've realized that's really important. More AWS tutorials is really needed. There's numerous of new programmers who want to learn AWS, but can't finish building anything because they get buried in documentation. I find there are a lot of high-level abstracted tutorials, but for the new services, there aren't a lot of detailed tutorials. For instance, an implemented cognito->gateway->lambda->dynamodb is…
I definitely agree that better tutorials and/or a simpler interface would makes AWS more accessible and user friendly. A YC S15 startup, Convox ( http://convox.com/ ), aims to "make AWS as easy as using Heroku." It looks really promising.
Convox member here. Thanks for the shout out.
This is definitely a goal of Convox: to remove as much AWS complexity as possible.
Our approach matches this guide to a tee. We are using CloudFormation to set up a private app cluster, as well as to create and update (deploy) apps. We are also using ASGs.
The instance utilization point is spot on too. The fist thing convox does to make this easy is a single command to resize your cluster safely (no app downtime).
Coming next is monitoring if ECS and CloudWatch and Slack notifications if we detect over or under utilization.
I strongly believe that these AWS best practices can and should be available for everyone. For anyone starting from scratch or migrating apps off a platform or EC2 Classic onto "modern" AWS.