Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

techcrunch.com

Post reply on HN
Print view View original

41–50 of 63 posts

Re: DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

#41

Post by [deleted] »

tedchs wrote: post #32
HorizonXP wrote: post #15

Earlier quoted context omitted.

This was useful, as I couldn't figure out how to add the cloud-config files I use for my Vagrant-base CoreOS cluster. Here's another question: If I have a droplet up and running already, anyone know how I might change it from Ubuntu to the new CoreOS image? I'd rather change it than create a new one to maintain the same public IP, or else I have to have my DNS records updated, which takes time, and is outside my dire…

CoreOS is not a drop-in replacement for Ubuntu; migrating to it requires re-deploying your services inside Docker containers. I would think you might want to run your service implementations in parallel and then cut over later. Can you not move IP's between Digitalocean droplets?

[deleted]

Re: DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

#42

Post by michaelsbradley »

The article How To Set Up a CoreOS Cluster on DigitalOcean[1] (written by a DigitalOcean employee) fails to mention what seems to me to be a serious security-related concern.

Since droplets with private networking enabled are on the same private network as other customers' droplets, then if "$private_ipv4" is specified for "addr" and "peer-addr" in cloud-config, isn't it critical that etcd be secured with TLS and client cert authentication?

See: CoreOS – Etcd: Reading and Writing over HTTPS[2]

I realize that delving into that aspect of coreos/etcd configuration is beyond the scope of an introductory "how to" article, but I believe that some strong mention should be given to this concern.

I made a comment[3] to this effect on DigitalOcean's website.

[1] https://www.digitalocean.com/community/tutorials/how-to-set-...

[2] https://coreos.com/docs/distributed-configuration/etcd-secur...

[3] https://www.digitalocean.com/community/tutorials/how-to-set-...

Re: DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

#43

Post by lukebennett »

fmdud wrote: post #5
thu wrote: post #4

Earlier quoted context omitted.

I still am very impressed with CoreOS but I find sad that they were forced to move away from their initially touted "only systemd+etcd". (No blame on them, that was expected as the Docker ecosystem is still young and very fast-moving; plenty of interleaved problems must be solved elegantly.) When you say that Docker+etcd is saner that other solutions, I wonder for how long this will remain the case as the field matur…

Well "Docker + etcd" still needs a surprising amount of tooling around it to be considered a viable production environment. CoreOS provides a lot of this and deis sits atop that. Deis is the best thing I've seen come out of Docker's DevOps gold rush.

"Deis is the best thing I've seen come out of Docker's DevOps gold rush."

I have to agree - Deis fills a lot of holes, although it still has some way to go.

Re: DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

#44

Post by knite »

michaelsbradley wrote: post #42

The article How To Set Up a CoreOS Cluster on DigitalOcean [1] (written by a DigitalOcean employee) fails to mention what seems to me to be a serious security-related concern. Since droplets with private networking enabled are on the same private network as other customers' droplets, then if "$private_ipv4" is specified for "addr" and "peer-addr" in cloud-config, isn't it critical that etcd be secured with TLS and cl…

This is a really great point. I may raise it on one of the CoreOS mailing lists.

Re: DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

#45

Post by kapilvt »

toomuchtodo wrote: post #31
kapilvt wrote: post #13

Earlier quoted context omitted.

its not about its shortcomings, its a database. your saying it replaces chef and opsworks. you're missing a few pieces to make that picture. ie. how do i update the load balancer as i add capacity to my web app, how do i setup a new db instance and open up the firewall rules to an app on a different host. etc. docker, and etcd are both nice pieces, but they don't make a complete picture. nor does fleet imo.

etcd is better compared to zookeeper (SOA orchestration, service discovery, etc) and etcd (after researching for the last two weeks for a production environment) is nowhere near as stable (ie non-beta/alpha) as zookeeper or Netflix's Eureka. Configuration management (puppet, chef, salt, ansible) is a completely different beats than service discovery, health management, etc.

agreed, etcd is like zookeeper, their both databases that you write applications against. No! neither does service discovery and orchestration out of the box. The applications you write on top of them do.

Re: DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

#46

Post by akbar501 »

michaelsbradley wrote: post #42

The article How To Set Up a CoreOS Cluster on DigitalOcean [1] (written by a DigitalOcean employee) fails to mention what seems to me to be a serious security-related concern. Since droplets with private networking enabled are on the same private network as other customers' droplets, then if "$private_ipv4" is specified for "addr" and "peer-addr" in cloud-config, isn't it critical that etcd be secured with TLS and cl…

The article you linked to provides security for etcd.

However, what is the standard approach for securing Docker container to container communication across hosts. For example from an app server to a DB server.

Is IPSec setup within CoreOS network layer, or is the security provided by Docker? If so, what are the options?

Re: DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

#47

Post by Donzo »

wastedhours wrote: post #11
STRML wrote: post #2

This is actually really big news for anyone running or interested in running a Docker-based PaaS system such as Deis or Flynn. DigitalOcean's cheap instances are a great match for Docker containers. As of Deis 0.8.0 it only runs on CoreOS, and I believe most other DIY PaaS systems are moving the same way. IMO Docker + etcd is a far more sane configuration than endless Ruby Chef scripts, or worse, Amazon OpsWorks.

I'm really out of touch with Docker and CoreOS, so please forgive my ignorance if this is a ridiculous Q: could this combo be used for spinning up machines with specific apps for thin clients? Or is it more about scaling one app?

I have another noob question. Would this be a good way to get redundancy for serving a website?

Re: DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

#48

Post by michaelsbradley »

akbar501 wrote: post #46
michaelsbradley wrote: post #42

The article How To Set Up a CoreOS Cluster on DigitalOcean [1] (written by a DigitalOcean employee) fails to mention what seems to me to be a serious security-related concern. Since droplets with private networking enabled are on the same private network as other customers' droplets, then if "$private_ipv4" is specified for "addr" and "peer-addr" in cloud-config, isn't it critical that etcd be secured with TLS and cl…

The article you linked to provides security for etcd. However, what is the standard approach for securing Docker container to container communication across hosts. For example from an app server to a DB server. Is IPSec setup within CoreOS network layer, or is the security provided by Docker? If so, what are the options?

I don't think CoreOS does anything special in this regard.

It should be possible via cloud-config to change the runtime config of the docker service[1], in which case one could set "--icc=false"[2] to enforce stricter rules about inter-container communication on a particular docker host (e.g. a coreos droplet).

[1] https://coreos.com/docs/launching-containers/building/custom...

[2] https://docs.docker.com/articles/networking/#between-contain...

EDIT:

Okay, I see you were asking about regulating network comm between containers on separate docker hosts, i.e. coreos instances.

That's a good question! I still don't think CoreOS addresses that concern in any special way at the level of iptables and routing (but I could be wrong). What it does give you is the ability to control service affinity with respect to your fleet "units". That way, you can be certain that docker containers which need to be "linked" in order to communicate properly (e.g. you have set "--icc=false") will run on the same host.

Re: DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

#49

Post by justizin »

kapilvt wrote: post #13
STRML wrote: post #12

Earlier quoted context omitted.

Just curious - what do you see as its shortcomings?

its not about its shortcomings, its a database. your saying it replaces chef and opsworks. you're missing a few pieces to make that picture. ie. how do i update the load balancer as i add capacity to my web app, how do i setup a new db instance and open up the firewall rules to an app on a different host. etc. docker, and etcd are both nice pieces, but they don't make a complete picture. nor does fleet imo.

While I agree that etcd and config management systems do not solve the same problems, and esp the same way, the use of etcd by all of your services does enable your examples.

For instance, there is a tutorial on using vulcand as your http load balancer, which is driven by configuration in etcd. A tutorial on the CoreOS site shows you how to use this to cleanly deploy a new version of an app as new versions of a container, and to rotate them into the LB and to rotate the old ones out.

You can also connect a script that controls firewall rules based on etcd, its' changes would be reflected almost immediately, rather than the splayed 30-60min period of seemingly-randomly-applying-changes you typically see with config runs of tools like chef and puppet.

I've been trying to figure out where tools like chef, fabric, and docker/coreos/etcd/fleet will play together, what the boundaries will be, etc.. In situations where I'm using Docker and CoreOS, I don't expect to use chef, but I'm not sure it will be used for things like database servers, which we typically dedicate and tune hosts for.

It would be nice to see something like chef running on CoreOS for, say, user management so that people who want to talk to, say, fleet aren't required to all ssh as 'core'.

Re: DigitalOcean Partners with CoreOS for Large-Scale Cluster Deployments

#50

Post by justizin »

kapilvt wrote: post #45
toomuchtodo wrote: post #31

Earlier quoted context omitted.

etcd is better compared to zookeeper (SOA orchestration, service discovery, etc) and etcd (after researching for the last two weeks for a production environment) is nowhere near as stable (ie non-beta/alpha) as zookeeper or Netflix's Eureka. Configuration management (puppet, chef, salt, ansible) is a completely different beats than service discovery, health management, etc.

agreed, etcd is like zookeeper, their both databases that you write applications against. No! neither does service discovery and orchestration out of the box. The applications you write on top of them do.

Right, and you can achieve service discovery, orchestration, and convergence driven by something like zookeeper or etcd, rather than by bulky config runs.

While I enjoy working with Chef, have had some pretty reasonable times with Puppet, these full-run tools do have issues sometimes where you introduce a narrow bug in your user management or some other code, and all of a sudden you can't update some random conf file that happens after it. I've also seen tools like capistrano and fabric bastardized to allow this sort of precise updating, but lose the cohesion of typical config-managed systems.

With something like etcd, certain problems are solved by not relying on static configuration. That is the paradigm shift, and it is similar to Hadoop. When I've built Hadoop systems with puppet, we simply wrote out the same configs and files on every machine, then chose which services to start, and via zookeeper things like primary / secondary failover take care of themselves.

Post reply on HN