Xmas Message from Ghana
http://nanabkay.blogspot.com/2015/12/pretty-esi-dorinda-shar...
AWS mistakes to avoid
81–90 of 276 posts
Re: AWS mistakes to avoid
#82I like CloudFormation. Unfortunately it is very unwieldy to write CloudFormation templates directly, and we're not about to start using the AWS CFN GUI editor! It seems like the assembly of the AWS ecosystem. Does anyone else have a favourite hammer for this particular nail? I'd love to have something better than our home-baked solution, but I'm yet to find anything which doesn't introduce other flaws, such as an inc…
Create the infrastructure manually and then use Cloudformer tool to generate the template based on the already created infrastructure. You can then edit the generated template to make it more maintainable and you have a nice reusable template.
Re: AWS mistakes to avoid
#83You know what I've realized that's really important. More AWS tutorials is really needed. There's numerous of new programmers who want to learn AWS, but can't finish building anything because they get buried in documentation. I find there are a lot of high-level abstracted tutorials, but for the new services, there aren't a lot of detailed tutorials. For instance, an implemented cognito->gateway->lambda->dynamodb is…
CloudAcademy provides several good ones.
Re: AWS mistakes to avoid
#84Also, dont use Dynamo unless you have a really good reason. "It scales better than MySQL" is not a good reason. Have fun migrating data and re-indexing constantly!
Can you explain these comments? I am currently developing a system using Dyanmo as the database, I need a JSON like structure as the documents need to be flexible as different documents will contain different keys. So I am interested to hear why you think Dyanmo is a bad choice.
Re: AWS mistakes to avoid
#85Earlier quoted context omitted.
Convox member here. Thanks for the shout out. This is definitely a goal of Convox: to remove as much AWS complexity as possible. Our approach matches this guide to a tee. We are using CloudFormation to set up a private app cluster, as well as to create and update (deploy) apps. We are also using ASGs. The instance utilization point is spot on too. The fist thing convox does to make this easy is a single command to re…
I hadn't heard of Convox before, but it sounds interesting. I'd like to use AWS more, but each time I tried to get into it I felt overwhelmed. I currently use PagodaBox a lot, which is great (most of the time) because it handles a lot of the complexity for me, but it can often be expensive. How does Convox compare to PagodaBox?
I've never used PagodaBox but it looks like a nice PaaS.
Convox has the same goal of a PaaS: to give you and your team an easy way to focus on your code and never worry about your infrastructure.
One big difference with Convox is that we accomplish this with single-tenant AWS things. You and your team's deployment target is an isolated VPC, ECS (EC2 container service), and ELB (load balancers).
If you're asking for a cost comparison, we're building Convox to be extremely cost competitive by unlocking AWS resource costs for everyone.
Its easiest to compare the cost of memory across platforms, though not always apples to apples...
The base Convox recommendation is 3 t2.smalls which is 6 GB of memory which costs about $100 / month. If your app can be sliced up into 512 MB processes, you can easily run 10 processes, which could be 2 to 5 medium traffic PHP apps on the cluster.
I'm finding PagodaBox pricing calculator a bit confusing but 6 512 MB processes, so 3 GB of memory, is $189.
Re: AWS mistakes to avoid
#86Earlier quoted context omitted.
If you'd consider something other than CloudFormation, there is also Hashicorp's Terraform. It has an AWS provider ( https://terraform.io/docs/providers/aws/index.html ) which creates resources and maintains the state in a file that you can store in version control ( https://terraform.io/docs/state/index.html ).
Strongly agree. Support for new AWS features hits Terraform much faster than CloudFormation (still waiting for CF support for AWS's managed ElasticSearch service that was unveiled two months ago--Terraform got it right away). Some of the critiques below are true... HCL is fine, but Terraform's interpolation syntax has a long way to go. That said, CF's JSON is way more painful to deal with. As for the other problems,…
> HCL is fine, but Terraform's interpolation syntax has a long way to go
Oh, HCL is fine, you say so authoritatively? Well then do me a solid and show me an if statement, show me a for loop. Because you're not building nontrivial, reusable infrastructural modules without logic. I know. I've tried. I've committed, between different projects and clients, somewhere around ten thousand lines of Terraform and probably half are copy-paste garbage because HCL is so crippled a tool.
It hurts me to say this at a deep and visceral level: Terraform's interpolation syntax makes freaking Ansible and its "no, really, it's totally cool, string templates for logic are awesome" look good.
> The right approach to any problems with Terraform is not to spread FUD about it like below, but to contribute code fixes.
Spread FUD? Oh, no no no, you can take your assertions of FUD and insert them somewhere uncomfortable, thank you very much. I wrote Terraframe[1] specifically to contribute back to the Terraform community, to make it better, and stopped (to create a different project) because I was stymied. By no documentation, by HCL JSON not actually working, and by no interest from the developers in any sort of dialogue about actually fulfilling the promises they themselves assert for their software. Between this and bugs that a trivial testing framework should catch (Why are you validating AWS resource names differently between point releases? Why are you changing that validation to be wrong? Why are you breaking my existing states when you've done this? Why did your tests not catch this before you pushed this out to your entire userbase?) I cannot take the project seriously as a tool for being used in infrastructure I care about. Because I don't trust them to take Terraform seriously, either.
Re: AWS mistakes to avoid
#87Earlier quoted context omitted.
If you'd consider something other than CloudFormation, there is also Hashicorp's Terraform. It has an AWS provider ( https://terraform.io/docs/providers/aws/index.html ) which creates resources and maintains the state in a file that you can store in version control ( https://terraform.io/docs/state/index.html ).
Terraform, as an idea, is brilliant. Mitchell and company isolated a hugely important need and tried to fill it, and I give them all the credit in the world for that. Cross-platform cloud provisioning? Gimme. But I cannot in good conscience not relate what a disastrous experience Terraform has been for me at both jobs and clients. Writing reusable code in Terraform is an exercise in frustration due to the extreme clu…
> Full disclosure: I'm building a much, much better provisioner for multi-provider cloud infrastructure. Neither of the projects I recommend are mine; mine's not done yet.
One of the convenient things about software that doesn't exist is that it doesn't have any bugs.
Let your software speak for itself when it exists; until then, this seems an undeserved critique of software, and a team, that is solving problems every day.
Re: AWS mistakes to avoid
#88Earlier quoted context omitted.
Convox member here. CloudFoundry is a really solid platform, but there is a very important distinction between CloudFoundry and Convox. Convox is a very thin layer on top of "raw" AWS. It gives you a PaaS abstraction but behind the scenes is well configures VPC, ECS, Kinesis, Lambda, KMS, etc. For those of us with no need to run on multiple clouds, using pure AWS is simpler, cheaper and more reliable than a middlewar…
how does convox scale when moving to non-trivial aws scenarios? if i have three autoscale groups running seven different apps with associated kinesis streams, s3 buckets, elasticache, rds and elasticsearch instances how cleanly does convox handle this? what about monitoring and reporting?
We are working to normalize the AWS scenario for all apps.
Every Convox cluster is an autoscale group managed cluster of ECS instances.
Every Convox app gets its own Kinesis stream for logs, ELB for load balancing, and S3 buckets for settings, build artifacts and encrypted environment. And the app processes are run via ECS.
So I'm confident we could handle the 7 apps in a single cluster, scale the cluster instance size and count, scale any individual app process type, and handle any individual app load balancing or log throughput.
You can provision some services like RDS with our tooling which make it really easy to link to apps. You can also bring your own services like elasticsearch or pre-existing RDS and set them in an app's environment to use it.
There are a couple monitoring tools built in.
We automatically monitor AWS events like ECS capacity problems and send Slack notifications.
You can also use our tooling to forward all your logs to Papertrail and configure your searching and alerting there.
More CloudWatch Logs and Metrics work is coming in the near future.
Re: AWS mistakes to avoid
#89Earlier quoted context omitted.
I am asking this question often. We are using very advanced CloudFormation in the open source Convox platform. https://github.com/convox/rack I have touched every corner of CF including lots of Custom Resources. Right now we are using the golang template tools and tests to generate our templates. But I have lots of needs and ideas for improving this. A CF template compiler and simulator should be possible, giving us…
One of the things that I thought would be neat is an open source CloudFormation that could work for multiple cloud vendors, possibly using a driver pattern. Also, you might want to update your HN profile with contact info.
Thanks, I updated my profile with contact info.
Terraform is awesome if you want an OSS project to manage multiple cloud vendors.
But I think that infrastructure change management is a really hard problem and the state of the art solution is how AWS runs CloudFormation as a managed service.
Once it's set up properly, it's amazing watching what CloudFormation can do. It can execute updating 20 instances to roll out a new AMI, and then roll the whole operation back on demand or if a failure happens. All with no application downtime in the cluster!
Re: AWS mistakes to avoid
#90I second the recommendation to use CloudFormation + packer + ELBs + auto scaling groups for web applications whenever possible, it just makes everything so easy and automatic. Of course, there's a learning curve and you pay a premium for all that automation, but in my experience it has been usually worth it so far.
I like going one step further and making deployments and AWS resources into reusable code using Troposphere: https://github.com/cloudtools/troposphere