Would you mind sharing more details about the Docker setup on AWS? "Our code deployment flow was straightforward, and mostly consisted of building a Docker image off our latest codebase and distributing it to our server instances, with the help of a Docker image server. We only needed to write a few short Chef scripts to download the latest Docker image build and start up a new Docker container." I was wondering what…
Store your Dockerfile in version control, either in its own repo with other Docker files or in the same repo as the app. Use Jenkins (or preferred build system) to fetch the Dockerfile, your app, build the Docker container, and then docker push to your own private docker registry. Bake your AMIs with your Docker image (tags are your friend). You don't want to have to scale and find out you're having problems fetching…
Migrating from Heroku to AWS using Docker
11–20 of 58 posts
Re: Migrating from Heroku to AWS using Docker
#12If you're looking at doing this, especially if you're moving from Heroku and/or using Docker, seriously consider AWS ElasticBeanstalk. http://aws.amazon.com/elasticbeanstalk/ In case you don't know, ElasticBeanstalk is the AWS Heroku but _a lot_ more flexible. It's basically a layer on top of EC2 and ELB. It takes care of creating auto-scaled EC2 instances to your specification and deploying your application to it (w…
Re: Migrating from Heroku to AWS using Docker
#13If you're looking at doing this, especially if you're moving from Heroku and/or using Docker, seriously consider AWS ElasticBeanstalk. http://aws.amazon.com/elasticbeanstalk/ In case you don't know, ElasticBeanstalk is the AWS Heroku but _a lot_ more flexible. It's basically a layer on top of EC2 and ELB. It takes care of creating auto-scaled EC2 instances to your specification and deploying your application to it (w…
Re: Migrating from Heroku to AWS using Docker
#14Re: Migrating from Heroku to AWS using Docker
#15> With Heroku, you lose fine-grained control and visibility over your servers. Installing custom software in your server stack is far from straightforward, and it’s not possible to SSH into a server to debug a CPU or memory issue. I found this annoying at first along with not being able to save files to the server. However, Heroku is forcing you to automate server setup so servers can be restarted, destroyed and crea…
The transition itself took one month. So far it's been several months and it's been very stable with pretty minimal maintenance. We did end up writing a number of Python scripts to help make the deploy process easy for developers (i.e. replicating most of the common Heroku features we used to use)
Re: Migrating from Heroku to AWS using Docker
#16If you're looking at doing this, especially if you're moving from Heroku and/or using Docker, seriously consider AWS ElasticBeanstalk. http://aws.amazon.com/elasticbeanstalk/ In case you don't know, ElasticBeanstalk is the AWS Heroku but _a lot_ more flexible. It's basically a layer on top of EC2 and ELB. It takes care of creating auto-scaled EC2 instances to your specification and deploying your application to it (w…
We're using EBS at StaffJoy and in the process of switching to multiple containers per EBS host. This means that a staging environment can have cheap redundancy (e.g. for zero-downtime deploys) by utilizing the same EC2 hosts, but also that queue workers (sporadic, processor-intensive scheduling operations) can split a processor with the more memory-intensive web app.
Getting EBS to work with the private docker registry was a bit of a pain because documentation was lacking - if you're having trouble with this, feel free to ping me. I'm considering a blog post on our deployment system, including easy rollbacks.
The problems we haven't figured out yet on EBS are streaming logs from hosts to another service (e.g. loggly/logentries/papertrail for realtime production error monitoring) and monitoring deploys (specifically to send a Slack notification that changes are fully deployed and to flush the Cloudflare cache).
(Note - here EBS = elastic beanstalk, not elastic block storage)
Edit: A final note - we ended up doing a Jenkins install instead of using CircleCI for build promotion purposes. We build a container, push to the docker registry, then deploy to a stage. We have a separate job that lets us deploy to prod. However, there's no rebuilding - it's just a button that takes a build number and tells EBS to deploy that existing build to the master cluster. For rollbacks - you just enter a prior build number and a prior build immediately starts rolling back We couldn't replicate this stage->master promotion/rollback in CircleCI.
Edit2: We chose to build the image ourself and register with the private registry instead of having AWS build it because we run our unit tests within the built container, so we can guarantee that the image that passes tests is the exact same as the one deployed to production. There was some concern that the image hash could differ - e.g. due to a pip download failure - if AWS built the image separately.
Re: Migrating from Heroku to AWS using Docker
#17If you're looking at doing this, especially if you're moving from Heroku and/or using Docker, seriously consider AWS ElasticBeanstalk. http://aws.amazon.com/elasticbeanstalk/ In case you don't know, ElasticBeanstalk is the AWS Heroku but _a lot_ more flexible. It's basically a layer on top of EC2 and ELB. It takes care of creating auto-scaled EC2 instances to your specification and deploying your application to it (w…
I am wondering how difficult it would be to "push to elastic beanstalk". It would be kind of cool to have some sort of git daemon running in the background, so you wouldn't necessarily need to use a third-party CI service. You could just push to a specific git remote, just like with Heroku.
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create...
Re: Migrating from Heroku to AWS using Docker
#18If you're looking at doing this, especially if you're moving from Heroku and/or using Docker, seriously consider AWS ElasticBeanstalk. http://aws.amazon.com/elasticbeanstalk/ In case you don't know, ElasticBeanstalk is the AWS Heroku but _a lot_ more flexible. It's basically a layer on top of EC2 and ELB. It takes care of creating auto-scaled EC2 instances to your specification and deploying your application to it (w…
I am wondering how difficult it would be to "push to elastic beanstalk". It would be kind of cool to have some sort of git daemon running in the background, so you wouldn't necessarily need to use a third-party CI service. You could just push to a specific git remote, just like with Heroku.
Re: Migrating from Heroku to AWS using Docker
#19Did you know your site was blocked in Indonesia? > This site is prohibited. The site is included in the list that should not be accessed through this network because contains porn, gambling, phising/malware, SARA or Proxy. > In accordance with Indonesian Government Regulation refers to Law Number 36 of Year 1999 on Telecommunications Article 21, "Telecommunication provider is prohibited from conducting business telec…
Re: Migrating from Heroku to AWS using Docker
#20If you're looking at doing this, especially if you're moving from Heroku and/or using Docker, seriously consider AWS ElasticBeanstalk. http://aws.amazon.com/elasticbeanstalk/ In case you don't know, ElasticBeanstalk is the AWS Heroku but _a lot_ more flexible. It's basically a layer on top of EC2 and ELB. It takes care of creating auto-scaled EC2 instances to your specification and deploying your application to it (w…
It's worth noting that Elastic Beanstalk just released support for multiple docker containers per instance. This takes EBS from being a novelty to being production-grade Docker orchestration. Previously, it ran a single docker container per instance which - while good for stage/prod parity and easy rollbacks - doesn't really take advantage of the core Docker advantages over a VM. Now, you can orchestrate multiple con…