Those using AWS ALB, how do you monitor your traffic in realtime? I want to aggregate host names, ip addresses, user agents in realtime.
Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?
51–60 of 68 posts
Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?
#52As a professional sysadmin, my go to reference on this is "Documentation Writing for System Administrators", from the Short Topics in System Administration series. https://www.usenix.org/short-topics/documentation-writing-sy... Also, this talk was very good: https://www.usenix.org/legacy/event/lisa08/tech/gelb_talk.pd...
It's worth the $5, I assume?
Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?
#53I'm going to mostly disagree with everyone here, much to my karma's detriment ;P I agree the end-goal should be infrastructure as code, and everyone here has covered those tools well. You also want monitoring across your infrastructure. Prometheus is the new poster-boy here, but the Nagios family, and many other decent OSS solutions exist as well. But you still need documentation. Your documentation should exist wher…
Windows sys admin here. OneNote is fantastic for IT documentation. I like that you can drag and drop a screenshot (no uploading to a wiki), store spreadsheets, word docs, PDFs, etc. and easily search for information via the built-in functionality.
We have used it for years and it has worked great for us.
Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?
#54I use One Note. But I also use the o365 Suite. Mediawiki is also good, but can be a bore to run another service for that. But in the end a textfile via notepad/nano is all you need, really.
OneNote over a text editor as you can drop in screen shots.
Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?
#55Aligning a VMWARE tagging strategy with a cloud tagging strategy is one of my current goals. Things like a full blown CMDB seem to always end in pain, lag, and orphaned records. I'm happy enough with something basic that spans on-prem + cloud.
Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?
#56By having your infrastructure defined in version control using some sort of domain specific language. For example, by using Terraform and only ever making changes to your infrastructure via Terraform (manual adding/editing of stuff in AWS/GCP console should be disabled so people can't do that). Then all changes to the infrastructure are clearly documented in version control with pull requests.
Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?
#57So, one problem I’ve seen with most infrastructure as code solutions and CMDBs is that they do a good job at the tactical level (more or less), and help you answer “how”, “where”, “what”, and maybe “when” questions (depending on how well they support orchestration), they typically do a bad job at the higher level strategic “why” questions.
So, why do you structure your lambda jobs accessing CloudWatch Logs that way as opposed to the other way? If you didn’t know that one way works and the other doesn’t, you wouldn’t be able to understand that question. And that might have domino effects on other parts of your system.
I haven’t found a good solution to documenting the high level strategic “why” questions, other than to just write down the questions and the answer, with reasoning, in some form of associated documentation — maybe in a wiki or something. But, of course, the underlying issues may change in the near future and invalidate the reasons for your decision. And the high level documentation doesn’t have any way to be compiled directly into the lower level implementation, so of course there is always the risk of drift.
I’m still looking for good solutions in this space.
Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?
#58anyone have any pointers for simple an API driven managment of DNS/DHCP? (like, I don't want to have to configure 1000 moving parts) typically this seems to fall into the 'roll your own' or 'giant lumbering enterprise behemoth' category that does 10 other things. I'm looking for the sweet spot.
[deleted]
Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?
#59anyone have any pointers for simple an API driven managment of DNS/DHCP? (like, I don't want to have to configure 1000 moving parts) typically this seems to fall into the 'roll your own' or 'giant lumbering enterprise behemoth' category that does 10 other things. I'm looking for the sweet spot.
PowerDNS has served me well and has a terraform provider. Many of the DNS SaaS offerings are also have decent APIs as does AWS route53
thanks
should probably look more closely at powerdns..
however this doesn't solve the dhcp side of things.
specifically not looking at SaaS since I want consistent deployment flexibility and potential for mixed scale/scope/environment deployments (devbox, lan, wan, mixture, yadda)
Re: Ask HN: How do you document and keep tabs on your infrastructure as a sysadmin?
#60anyone have any pointers for simple an API driven managment of DNS/DHCP? (like, I don't want to have to configure 1000 moving parts) typically this seems to fall into the 'roll your own' or 'giant lumbering enterprise behemoth' category that does 10 other things. I'm looking for the sweet spot.
At any reasonable scale you typically wouldn’t use plain DNS if you have to do that kind of figuration. It would be done with a service discovery service which handles SRV records. That being said route53 has a reasonable management API.
thanks - should have mentioned specifically not looking at cloud services
(e.g. self hosted, but without needing 5 different polyglot microservices and a service managment layer and 32GB of ram just to keep the whole mess running)
I see 0% need for this complexity in many cases on the presentation side - and if faster response is required internally, the same API IF can be used for service discovery or side-chain announcements, etc can be bolted on on a per-application basis if desired.
I also see 0% need for this to be a cloud exclusive domain - e.g. hybrid scope/location deployments, etc.