Live data from Hacker News

Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?

news.ycombinator.com

1–10 of 68 posts

Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?

#3

I'm not sure I understand your question fully? You write documentation, like you do anything. And configure everything with code, so you can go read it (Terraform, Chef/Puppet/Ansible, etc).

OP is probably looking for someone to go a little further into detail on exactly what you just said.

Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?

#5
I've used https://www.racktables.org with pretty good luck. It's PHP, which wouldn't be my first choice, but I've largely been able to make it do what I want.

If you want something more clever; say keeping track of asset values etc, you'll want a CMDB. Google around and you should find something that fits your needs. We used SeviceNow in a previous life.

Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?

#7
- keep inventory in a DCIM (we use Netbox)

- configure everything as code (we use Ansible for the infrastructure up to OS level, Kubernetes w/ Helm for applications), have it read the values from the DCIM so that the DCIM remains the single source of truth (we need to still get better on this part....)

Links: https://github.com/digitalocean/netbox https://www.ansible.com https://www.kubernetes.io

That's at work. At home, I do much of the same, except that maintaining a DCIM is excessive for 2 VPS and a home network of 3 boxes.

Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?

#8
anyone have any pointers for simple an API driven managment of DNS/DHCP?

(like, I don't want to have to configure 1000 moving parts)

typically this seems to fall into the 'roll your own' or 'giant lumbering enterprise behemoth' category that does 10 other things. I'm looking for the sweet spot.

Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?

#9
For me/us, it's a combination of infrastructure-as-code and metrics reporting/logs. Most of our boxes are swapped out on a weekly or more frequent basis, so the only accurate picture of what's running right this moment is the graphs built by the metrics collection tools. The only accurate picture of what's running on those boxes is the code which built the infrastructure.

There are a couple of exceptions, but those are actively being brought under the above model (mostly because they are effectively invisible, and the existing documentation for them is... incomplete).

Any documentation outside of that is stale in a few hours, and obsolete in a week.

Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?

#10
Back when I was put in charge of IT Lifecycle management for my Army unit (not by choice - "Hey, you've got a CS degree, so anything tech related goes to you"), I kept it all in an Access Database, and ran off a report occasionally to update my smartbook (3-ring binder full of stuff that my boss would frequently ask about during meetings). Granted this was back in the early 00's.
Post reply on HN