Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS • Hacker News Board

Re: Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS

#71

Post by juliangregorian » Wed, Jun 17, 2015, 6:38 PM UTC

whalesalad wrote: post #59

I'm interested in hearing more about how you use this in terms of development lifecycle. Does a container image get created for every release of your app? I've always wondered about the more correct approach to this. This is how I currently use Docker: 1) Custom base image with all the things my company needs like supervisord, libpq, etc.. 2) Custom per-service base images like ones with Java for our Clojure services…

Why not just add a Dockerfile to your git repo and add a `docker build` step to your deploy?

Re: Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS

#72

Post by MCRed » Thu, Jun 18, 2015, 1:04 AM UTC

peteretep wrote: post #42

floridaguy01 wrote: post #7

aws is silly expensive. Why didnt you build this on top of digitalocean? Digitalocean is so awesome right now. They dont even charge for bandwidth overages.

"But they'd be wrong! Truth is, I thought it mattered. I thought that [the marginal hosting costs between cloud service providers] mattered. But does it bollocks. Not compared to how [developer productivity and costs] matter."[0] [0] https://www.youtube.com/watch?v=W3tFZxhVaXo

Yes, and going with AWS kills productivity because you have to learn AWS specific APIs, you're locked into their system and then you have to re-work your stuff to be hosted elsewhere when you outgrow AWS.

More than one startup has been killed purely by AWS hosting costs in the past 5 years.

Re: Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS

#73

Post by loki77 » Thu, Jun 18, 2015, 4:46 AM UTC

scanr wrote: post #56

LunaSea wrote: post #51

Earlier quoted context omitted.

Any reason(s) for switching to Vulcand rather than NginX ?

First off, nginx is awesome and you can do all the things we did with vulcand with nginx, so it was just a question of friction. The reason we went with vulcand is that it natively supports what we wanted to do i.e. route to micro-services based on dynamic etcd driven configuration. To do the same thing in nginx (at the time), we would have either had to use confd or custom lua.

We actually looked at VulcanD in an older version of Empire. When we decided to use a routing layer with this version of Empire, rather than just letting Empire/ELB expose each service (mostly because it is a lot easier for us to later shut off public access to each service) we threw together nginx because it was so simple.

I think at this point everytime we move a service we add like 5 lines to an nginx config, re-deploy the router in Empire, and the service is exposed.

The internal 'service discovery' makes this a lot easier, since we just have to tell nginx to route to " rel="nofollow">http:// - no domain, no port, nothing more than the app_name thanks to DNS/resolv.conf search path & ELB stuff.

Re: Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS

#74

Post by loki77 » Thu, Jun 18, 2015, 4:51 AM UTC

UserRights wrote: post #54

How to autoscale with this?

So at this point Empire itself doesn't deal with things like Autoscaling. That said, the Demo Cloudformation template (and the bootstrap script that kicks it off easily for you) make use of Autoscaling groups for the instances that containers are being run on.

So, in theory you could autoscale just like you always would. Monitor stats for a host, if a bunch of them start to run low on resources, kick off an autoscaling event.

That said, there's been quite a bit of talk about integrating Empire with Autoscaling, so that when, say, ECS couldn't find any instances with resources free for a task, Empire could kick off the autoscaling events for you. Could be pretty awesome :)

Re: Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS

#75

Post by peteretep » Thu, Jun 18, 2015, 4:59 AM UTC

MCRed wrote: post #72

peteretep wrote: post #42

Earlier quoted context omitted.

"But they'd be wrong! Truth is, I thought it mattered. I thought that [the marginal hosting costs between cloud service providers] mattered. But does it bollocks. Not compared to how [developer productivity and costs] matter."[0] [0] https://www.youtube.com/watch?v=W3tFZxhVaXo

Yes, and going with AWS kills productivity because you have to learn AWS specific APIs, you're locked into their system and then you have to re-work your stuff to be hosted elsewhere when you outgrow AWS. More than one startup has been killed purely by AWS hosting costs in the past 5 years.

    > going with AWS kills productivity because you have to
    > learn AWS specific APIs

Products I use:

* Redshift. It's Postgres's API, and I didn't have to learn how to manage petabyte-scale clusters

* EC2. It's Ubuntu. Or CentOS. Or whatever. You choose! Except no messing about with my own virtualization or hardware agreements or going ot the datacenter.

* RDS. It's whichever database you want it to be! Only it scales! And backsup! For free!

* ElastiCache. It's Redis!

etc. etc. etc.

Learning those dang AWS-specific APIs, eh? Who'd do it?

     > has been killed purely by AWS hosting costs

Then they planned badly, because AWS prices consistently go down over time. If your "business" goes under because it becomes popular, then your marginal cost per user is negative, and you're running a charity for the benefit of your users, not a business. Blaming the demise of a company whose business model is giving out free icecream on the cost of icecream is missing the point a little.

Re: Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS

#76

Post by stephenr » Thu, Jun 18, 2015, 5:39 AM UTC

nickpsecurity wrote: post #66

stephenr wrote: post #33

Earlier quoted context omitted.

I understand your point about control completely! My highest priority for businesses making these choices is usually slightly more pragmatic, and focuses on avoiding provider lockin - something that you can install and run on your own local hardware, you can also (in most cases) install and run on co-located hardware, on rented hardware, on traditional rented virtual hardware (i.e. VPS) or on "flexible" rented virtua…

That's a very pragmatic philosophy. I'm especially impressed at your unusual focus on vendor neutrality as a lack of it costs many companies millions in long run (see IBM & COBOL). Since I mostly avoid clouds, I'm not up to date on that end. I'd like to attempt your style of things as an experiment in the future, though. Do you have a resource or resources for what components, strategies, or platforms are best for th…

Thanks. It doesn't always work out - clients/managers often seem to have "all the cool kids are using it" and/or "but it's the cloud, everyone uses the cloud now" mentality, but I try.

I should also emphasise that I'm mostly talking about infrastructure level "lock in" here - e.g. the artificial lock-in AWS creates for their Load Balancer and/or Elastic IP service by giving VMs new IPs on reboot, etc.

I definitely prefer Open Source solutions, but I'm one step more pragmatic in that space too - if a piece of locally-installable but proprietary software does the job and works with open standards (e.g. if you want to use self-hosted atmail) I'm less worried/vocal about that than if you say you want to use Gmail or whatever, but I'd still try to suggest a more open option.

In terms of resources, no sorry I don't have any single resource to go on, besides a basic rule/test:

Can I demonstrate the full stack being implemented, using one or more laptops (e.g. using VMs) on a plane or cruise ship? You could equally say "can i test the full stack while the WAN is disconnected" but that doesn't sound as fun!

I'm actually building my new business around this basic idea - giving smaller companies a better option to keep more control of their tech without the need for a full-time sysadmin (which is often financially impossible even if they wanted to). I genuinely believe the vast majority of things most businesses want/need to achieve can be done with existing Open Source software, its just usually not particularly easy to setup the various pieces, and make them work together.

Re: Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS

#77

Post by benblack » Thu, Jun 18, 2015, 5:55 AM UTC

Veratyr wrote: post #21

helloiamaperson wrote: post #12

Earlier quoted context omitted.

http://www.openshift.org/ or http://cloudfoundry.org/index.html , both used in production by f500 companies

Cloud Foundry requires a ton of (compute) overhead to get set up. It's very much intended for large projects. Hell, you need a separate VM just to install it. For anyone looking at a Dokku alternative, Cloud Foundry isn't one. Openshift is nice though.

Lattice is a more minimalist approach, for those who desire the production hardened core of Cloud Foundry but prefer to bring their own PaaS components or simply experiment.

http://lattice.cf/

Re: Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS

#78

Post by nickpsecurity » Thu, Jun 18, 2015, 4:10 PM UTC

stephenr wrote: post #76

nickpsecurity wrote: post #66

Earlier quoted context omitted.

That's a very pragmatic philosophy. I'm especially impressed at your unusual focus on vendor neutrality as a lack of it costs many companies millions in long run (see IBM & COBOL). Since I mostly avoid clouds, I'm not up to date on that end. I'd like to attempt your style of things as an experiment in the future, though. Do you have a resource or resources for what components, strategies, or platforms are best for th…

Thanks. It doesn't always work out - clients/managers often seem to have "all the cool kids are using it" and/or "but it's the cloud, everyone uses the cloud now" mentality, but I try. I should also emphasise that I'm mostly talking about infrastructure level "lock in" here - e.g. the artificial lock-in AWS creates for their Load Balancer and/or Elastic IP service by giving VMs new IPs on reboot, etc. I definitely pr…

That makes sense. It has been done before to a degree. For inspiration, look at Net Integrators Nitix appliance [1]. A UNIX system that was easier to configure, self-managing, largely auto-configed, partly self-healing, automatic backups, HA support, had most applications and a UI to integrate their configuration. It was selling well despite being priced above most SOHO servers. As often with good tech, a big firm (IBM) gobbled it up and rolled the tech into their own stack (Lotus).

A stack like you describe with good traits of Nitix-like solutions could be great for businesses not wanting much IT overhead. Might spread like wild fire so long as you don't sell out or balk over patent suits.

[1] http://www.pcmag.com/article2/0,2817,1734766,00.asp

Re: Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS

#79

Post by stephenr » Fri, Jun 19, 2015, 6:04 AM UTC

nickpsecurity wrote: post #78

stephenr wrote: post #76

Earlier quoted context omitted.

Thanks. It doesn't always work out - clients/managers often seem to have "all the cool kids are using it" and/or "but it's the cloud, everyone uses the cloud now" mentality, but I try. I should also emphasise that I'm mostly talking about infrastructure level "lock in" here - e.g. the artificial lock-in AWS creates for their Load Balancer and/or Elastic IP service by giving VMs new IPs on reboot, etc. I definitely pr…

That makes sense. It has been done before to a degree. For inspiration, look at Net Integrators Nitix appliance [1]. A UNIX system that was easier to configure, self-managing, largely auto-configed, partly self-healing, automatic backups, HA support, had most applications and a UI to integrate their configuration. It was selling well despite being priced above most SOHO servers. As often with good tech, a big firm (I…

Thanks for the vote of confidence and the reference!

Re: Introducing Empire: A Self-Hosted PaaS Built on Docker and Amazon ECS

#80

Post by parasubvert » Fri, Jun 19, 2015, 7:00 PM UTC

merb wrote: post #38

helloiamaperson wrote: post #27

Earlier quoted context omitted.

> Hell, you need a separate VM just to install it. I'm not sure why that's a problem. If you want something that's actually like heroku like in terms of uptime and what not, you need something that can manage the health of the cluster. Dokku's cool, but it doesn't make sense for anything you actually need to depend on. If it doesn't make sense to pay for the overhead of running your own paas, just use Heroku instead.

Heroku isn't cheap. And to run a small PaaS like alternative there are many many more solutions that are way better and faster and more secure than Cloud Foundry. Hell I mean did you even read the requirements of Cloud Foundry, it takes a hell lot of everything just to manage a few boxes. Also CF comes from Pivotal, listen to the talks they made, they say, start out small, however starting out small with CF is just i…

"are way better and faster and more secure than Cloud Foundry"

Like how? Better, okay it's in the eye of the beholder. But secure? Faster? Shenanigans.

IF you want to just get started with Cloud Foundry, you install Lattice, it's one VM. In an HA setup (not the target but possible), it's 5 VMs.

Cloud foundry is about 20 VMs, and is what happens when you say "and then I want..." About 5 times: multi tenancy, role based access control, auto recovery health management, service brokerage, multiple DNS domains, app staging, etc. wherein people will otherwise just build on their own ...