Show HN: Enroll, a tool to reverse-engineer servers into Ansible config mgmt

My thoughts exactly. As someone who has generally learned better and faster through labs or real world work, this is exactly how I intend to teach myself Ansible while also migrating some stuff to containers: throw at my current VMs, identify configs, and then migrate or enroll accordingly.

A question I didn't see answered on the Web site or in the comments... How does it determine what is "state"? Packages are easy, you just get a list of those with the package manager. I guess systemd lists services. But what about everything else? Is it just walking the whole file system and checking every file it finds to see if a package owns it? What about files and symlinks created by postinst scripts? Does it have special "knowledge" about certain services like Apache which have their own Ansible modules?

I don't know if this touches networking or not but I've always deliberately avoided configuring (most) networking on Ansible after having shot myself in the foot too many times.

I like this in theory and will give it a shot but I'm wary of getting bled out by all the sharp corner cases...

Thank you, Jeff! What an honour! Let me say that nothing compares to your catalogue of Ansible roles that I have benefited from for many years, which are far better written than anything Enroll generates :) I intend for Enroll only to be a 'rapid grab-it-all', good for certain use cases.

Appreciate it!

Thanks usrme! Good questions.

I run QubesOS as my workstation, so it's been really beneficial here, because Qubes is all VMs and templates. I've been 'harvesting' one Qube VM and then building another and running the manifest on the second machine. It's been working very well to align it with the first machine. Most of my testing has been visually watching the Ansible play install/configure things that I expect to see occur.

Where it falls down:

1) Systems that are so old, the packages that it detected as being installed, are no longer 'installable' on a new system (e.g the apt repositories no longer have those packages, they just did at the time of the original install)

2) Packages that were installed not via an apt repo but, say, dpkg -i (of a .deb file). Slack Desktop is a good example. Obviously the deb is not in the harvest, so it fails there.

So, there'll always be corner cases, but assuming everything that you have installed on the old/obscure machine is still something that can be installed via the usual methods, it should be okay. (If you are running a system that is so old, its packages are no longer available upstream, it's probably time to let it go! :) )

You'll want to use --exclude-path to ignore some server-specific stuff, perhaps, such as static network configuration etc. And of course, you can also comment out whatever roles are superfluous, in the playbook, before running it.

Always use --check with Ansible first just in case.

Thanks for the comment and questions! Very wise.

Let me break it down as to what harvest does:

1) Detects the OS and its package backend (e.g dpkg vs rpm etc)

2) Detects what packages are installed

3) For each package, it tries to detect files in /etc/ that have been modified from the default that get shipped with the package . It does't walk the whole filesystem, it looks straight in /etc for stuff here.

4) It detects running/enabled services and timers via systemd. For each of these, it looks for the unit files, any 'drop-in' files, environment variable files, etc, as well as what executable it executes, and tries to map those systemd services to the packages it's already learned about earlier (that way, those 'packages' or future Ansible roles, can also be associated with 'handlers' in Ansible, to handle restart of the services if/when the configs change)

5) Aside from known packages already learned, it optimistically tries to capture extra system configuration in /etc that is common for config management. This is stuff like crons, logrotate configs, networking settings (as you noted!), hosts files, etc.

6) It also looks for other snowflake stuff in /etc not associated with packages/services or other typical system config, and will put these into an etc_custom role

7) Likewise, it looks in /usr/local for stuff, on the assumption that this is an area that custom apps/configs might've been placed in. These go into a usr_local_custom role.

8) It captures non-system user accounts, their group memberships and their .ssh/authorized_keys

9) takes into account anything the user set with --exclude-path or --include-path . For anything extra that is included, it will put these into an 'extra_paths' role. The location could be anywhere e.g something in /opt, /srv/ whatever you want.

10) writes the state.json and captures the artifacts

So yes, you're right - it does capture stuff that many people might want to exclude if they are going to use the manifests to build other machines from that harvest (as opposed to just rebuild the same machine itself).

But you can use --exclude-path /etc/network and so on to skip the bits you don't want. You also can always comment out from the playbook.yml or delete certain roles it generates once you've run the 'enroll manifest'.

It doesn't have any knowledge of Ansible Galaxy roles/modules etc. It generates all the roles itself. I admit, many of the existing roles out there are a lot more pleasant/easier to read (especially Jeff Geerling's). I still use those myself day to day. A lot of configs out there are also good candidates for being Jinja templates with abstracted vars for separate hosts. Enroll does use my companion tool JinjaTurtle if it's installed, but JinjaTurtle only recognises certain types of files (.ini style, .json, .xml, .yaml, .toml, but not special ones like Nginx or Apache conf files which have their own special syntax).

I consider Enroll to be a good 'quick, grab it all, so I can sleep at night' method, perhaps best for DR purposes.

In terms of safety measures: it doesn't traverse symlinks, and it has an 'IgnorePolicy' that makes it ignore most binary files (except GPG binary keys used with apt) - though if you specify certain paths with --include-path and use --dangerous, it will skip its own policy. See https://git.mig5.net/mig5/enroll/src/branch/main/enroll/igno... .

It will skip files that are too large, and it also currently has a hardcoded cap of the number of files that it will harvest (4000 for /etc/ and /usr/local/etc and /usr/local/bin, and 500 per 'role'), to avoid 'bomb' situations.

I think your caution is very warranted and wise, and I highly recommend to always use --check with Ansible when/if you get to applying the playbook!

Thanks again.