Habitat — A new approach to automation

The article link points directly to the Habitat demo. For those of us that don't already know what Habitat is, a better place to start might be the about page: https://www.habitat.sh/about/why-habitat/

Habitat seems like an entirely new Chef product to enable what they're calling Application Automation.

Relevant quote: "[...] automation must travel with the application, rather than be provided by the platform. Everything the application needs, from build dependencies, run-time dependencies, configuration, dynamic topologies, deployment strategies, secrets management, security auditing, and so on belongs with the application, because we're doing it for the application. Runtime and infrastructure layers exist to support the application, but the efficient building, deployment, and management of the application are decoupled from those layers."

Incredibly ambitious.

Containerization is disrupting a lot of their current market, so they need to adapt in some fashion. Given the fact that the only hammer they have at hand is ruby systems automation tooling it is not a big surprise they are aiming at these targets.

Unfortunately, IMHO, the scope is a mile wide and an inch deep in terms of what chef brings to the table here. I can walk down that list and either think immediately of another tool that is dedicated to the specific task or the needs are so slight that a few ansible or bash scripts take care of the need. Sounds like they are aiming to occupy some space in the general "tooling & orchestration" arena, but it will be an uphill climb.

As nice as the 'console' on the page is, the fact the window gets cleared when you click to go to the second step makes it look like it lost what you did on the first page. To me that makes it look like it's broken - the steps should maintain the state and build on each other, not start over each time.

Comments based on the most trivial skimming of the docs, so I'm probably wrong about everything.

> Incredibly ambitious.

It is, but ambition is necessary if you want to build distributed production systems. BOSH[0] does something similar in providing an all-or-nothing approach to configuring distributed systems built out of heterogenous software components spread across multiple machines.

Note here that the key is the heterogenous part -- if you have an undifferentiated flock of software, you can rely on a different abstraction to deploy them (anything from PaaS to a hand-rolled thing, though I don't recommend hand-rolling this stuff). But very few distributed systems are designed to rely on emergent behaviour. In fact that's usually what we're trying to avoid.

Where they appear to differ is that BOSH is quite strict about lifecycle -- you can't "truly" update configurations on-the-fly. You update the relevant YAML and then ask BOSH to update the world for you. It seems like a hassle, but if you couple it with git and Concourse pipelines, you get a complete, externalised history of every system configuration change you've ever made.

We already know that this top-down model of heterogenous distributed systems configuration management works better than trying to extend it from the machine up (as Cfengine and its various successors tried to do). BOSH has been deploying large systems for years and years. It was originally designed to deploy and upgrade Cloud Foundry without downtime, something it does with aplomb.

What I really like about Habitat over BOSH is the way it exposes the available knobs and dials. That kind of interactive discoverability is super helpful sometimes.

I was, however, struck by this claim:

> This architecture allows us to avoid the production cliff problem: the closer you get to production, the more difficult it is.

This is literally the problem that PaaSes solve for you. I've seen companies without PaaSes, with home-rolled PaaSes and with Cloud Foundry. In the latter case, the biggest obstacle is usually getting VPN access. And there are other PaaSes apart from CF. Habitat seems to be doing a kinda-sorta PaaS here (note the "Supervisor"), but without most of the feature plumbing that makes PaaSes operationally useful: routing, logging, service injection and the like.

Disclaimer: I work for Pivotal, we do the majority of the engineering on both BOSH and Cloud Foundry.

[0] https://bosh.io/

This is an excellent point - kind of goes against my earlier comment. Maybe they have to move this way because that's their only choice - if they don't move into automating this, containers will disrupt them (or already has)