Live data from Hacker News

How I spend my first 5 minutes on a server

plusbryan.com

171–180 of 355 posts

Re: How I spend my first 5 minutes on a server

#171
post #95

Why install fail2ban? You already have SSH password auth disabled, and you only allow SSH connections from your office. Won't this just risk banning your own office if someone's SSH client is misconfigured?

Seconded.

Autobanning always causes trouble if enough people use the server from one IP address.

Re: How I spend my first 5 minutes on a server

#172
post #109

Earlier quoted context omitted.

Deployment tools like fabric are imperative, and configuration management tools are declarative. With configuration management, you define the final state you want the server to be in, and it will do whatever is needed to get it into that state. Some or all of the parts might already be done, and it won't change the parts that are already correct (the declarative configuration is idempotent). Deployment tools just ex…

Thanks for the clarification. I think I get the theory a bit better now. I'm still struggling with seeing some practical examples of those differences. for example, `apt-get` is pretty much idempotent already, no? if I run `apt-get install -y ` x 1000 times in a loop it won't install it 1000 times... Fabric does give you the building blocks for those kind of checks elsewhere, such as `exists`, `contains`, `append` (w…

apt-get is most definitively not idempotent without a lot of care. It will upgrade packages if a new version is available, and such upgrades can have widespread implications.

Otherwise I agree with you - Puppet/ Chef etc. are overly big on ceremony for me. We "only" use about 100 VM's on about 20 physical servers at the moment, so it's a "simple" setup.

I think where people get the most value out of systems like Puppet and Chef is if a lot of their servers or VM's have simple enough requirements that they can largely use "off the shelf" community provided configurations. The more customisation you need, the more the leve of ceremony starts biting you. The more you can rely on ready-made scripts, the more those apps help you because of the size of their communities.

Re: How I spend my first 5 minutes on a server

#173

Firstly, a nice checklist. Easy actionable steps, repeatable, and pretty much most of what you need. Secondly, you are about 4-5 hours away from learning puppet (or Chef) and making this checklist into actual code. Thirdly, you now have a checklist of items that you can use in a job interview if you get the oppertunity to gain a new-hire or an intern. Lastly, good on you for submitting this to a peer-review on HN. We…

We can be picky, but with good reason as security is an exact science and a costly one to get wrong. There's so much conflicting and out right bad advice posted online these days that sometimes it takes a picky community to help clarify the best practices. For what it's worth, some of the advice given in that article was worth mentioning (eg fail2ban, it's a great tool). But the shared account suggestion was the comp…

Picky is good! We never grow without constructive critisim. And a community of practice may yeild better results than an individual (at least for simple things).

> I'd recommend people read that article with a degree of scepticism before rushing onto any boxes they might administrate.

Agreed. But the same could be said of everything; hackers and engineers, empiricists both.

Some of the advice in the article would seem apporpriate to someone who had never worked with another senior engineer. No fault of the OP.

System engineering and security are as much a learned craft as a science; a dialectic between sand castles (if you will).

Re: How I spend my first 5 minutes on a server

#174
post #33

Earlier quoted context omitted.

Don't do this. It adds almost no extra security and makes it hard for routers that prioritizes port 22 traffic as interactive.

Sure, it'll not stop dedicated manual intrusion attempts, but it will actually prevent a ton of automated bots from even just trying to connect with common passwords through SSH.

>Sure, it'll not stop dedicated manual intrusion attempts, but it will actually prevent a ton of automated bots

Doesn't take long to port scan a server.

Re: How I spend my first 5 minutes on a server

#175

Earlier quoted context omitted.

IMHO, yes. At least on Ubuntu, it's never been too clear to me how I should save my rules so that they come back on startup. The ufw man page is pretty decent.

I've not tried anything complex with UFW so I still use iptables on my bastion host that handles my vpn tap. It's not terribly complex to make rules come back on startup (but probably more involved than one would hope). For anyone else that followed the thread to this point- this advice on bringing iptables back up on reboot worked for me http://rackerhacker.com/2009/11/16/automatically-loading-ipt... YMMV

This is how I run iptables on a sufficiently large network of machines.

The advice is not complete. IPv6 is real and really works most of the time these days. Back up your ip6tables to a file too. I like /etc/firewall-4.conf and /etc/firewall-6.conf but it's down to preference.

Know about iptables-apply too, lest you be caught unaware.

Re: How I spend my first 5 minutes on a server

#176
post #133

It saddens me each time I see a security best practice guide that suggest turning off ssh access for root. Its a very useful feature, and the security industry should focus on the security problems rather than removing features without thinking about the actually benefits of doing so. Sysadmins with root access should be able to handle a random 8 character or longer password, and that number is large enough for a sec…

You fail to understand that true security is not about passwords but control . Disabling remote root login, 1. cheaply reduces attack surface 2. necessitates assigning administration rights to specific persons and roles. 3. greatly increases command visibility and value of audit trails. 4. combines extremely well with disabled terminal shell over SSH 5. can reduce (unnecessary) system resource usage. Etc. There are v…

You fail to understand that true security is not about passwords or control; true secrity is about risk mitigation.

Control is an ignorant tyrants last redoubt. Security is only about: identifying risks and dealing with those risks. All other messures are worthless as actual security objectives.

I am not arguing with you in priciple, only in particular. All of your points are well taken, to reduce risk, but not to control.

Re: How I spend my first 5 minutes on a server

#177
post #165
post #71

Earlier quoted context omitted.

For #4, wouldn't the only change when someone leaves the organization be to remove their key from authorized_keys for the shared account? Why would anyone else have to be updated?

If they have had access to a shared account, how much work are you willing to put in to verify that there's nothing in that shared account that will get executed by another user later that will quietly reinstate a key? A reason to have separate accounts is that not only do you terminate access, you also have an easier time ensuring that less of what that person had access to could have been compromised. (This of cour…

But if you give people any access to any system surely this is a concern. As a software dev, maybe I've inserted something into one of the build scripts that quietly re-opens my backdoor to the source control server...

I haven't, but if you assume actually malicious users you're probably going to end up with something so locked down it's useless. Aren't you?

Re: How I spend my first 5 minutes on a server

#178

I do not recommend unattended upgrades. Every upgrade without any exceptions have the possibility to destroy any working system or your applications. Test and deploy would be wiser. By the way, what the heck developers do on production systems?

Bear in mind that in small company, developer == dev op == sys op == network manager = testers == toilet cleaner. i.e. they do everything, dev or production.

As regards to unattended upgrades, again in small companies the choice is usually between them or no upgrades (no time for the test/deploy cycle across all platforms in use etc., less frequently used systems get forgotten and so on). As the article points out, its safest just to go for security updates only. If the security update breaks things, then yes, your system is down until you fix it. If a hacker breaches your system, your system is down until you re-deploy it, and then you have to deal with the PR fallout of lost credentials and data etc. Its only anecdotal, but I've had no problems with unattended security updates on my ubuntu boxes in the 5-ish years. Security breach attempts are a daily occurrence. It's nice if its not an either/or choice, but for many it is.

Re: How I spend my first 5 minutes on a server

#179

Couple of related questions: 1) Many people seem to be recommending Puppet / Chef. How many servers or installs do you need before this is a good ROI? (Over using odd bash scripts or cPanel/WHM) 2) Am I right in thinking kernel updates don't get applied until the server is rebooted? If so, how / when do you manage this?

Re 2 yes that is true. You need to build this in to your maintenance.

Do you just reboot all machines every month or so or do you keep track of when kernel security patches come out some how?

Re: How I spend my first 5 minutes on a server

#180
> vim /home/deploy/.ssh/authorized_keys [and] Add the contents of the id_rsa.pub on your local machine and any other public keys that you want to have access to this server to this file.

Cool trick: use ssh-copy-id from the client machine.

From the man page:

    ssh-copy-id - install your public key in a remote machine's authorized_keys
It is much easier than editing the remote .ssh/authorized_keys file since copying and pasting the key is error prone due to extra new lines typically added by the terminal emulator.
Post reply on HN