Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SSH Emergency Access

smallstep.com

Post reply on HN
Print view View original

41–50 of 74 posts

Re: SSH Emergency Access

#41

Post by rsync »

noodlesUK wrote: post #2

I think another thing we might want to learn about is how to sound the alarm when the break glass is used. Is there an easy way of doing that with SSH? Running a command to page the ops/security team when a server receives a login attempt with an emergency credential?

"I think another thing we might want to learn about is how to sound the alarm when the break glass is used. Is there an easy way of doing that with SSH?"

Yes - quite simple and old-fashioned, actually ...

I have this line in the SSH users' .login file:

  /usr/local/sbin/sms 4153331111 4158882222 "USER LOGIN TO XXX - $DATE" >& /dev/null
... where the 'sms' command, above, is a shell script I wrote to call twilio messaging with the curl command. A very simple example of that would be:

  curl -X POST -d "Body=$msg" -d "From=$from" -d "To=$to" "https://api.twilio.com/2010-04-01/Accounts/$accountsid/Messages" -u "$accountsid:$authtoken"
... and this works like a charm.

Alternatively, you could rick-roll your on-call sysadmin:

  /usr/local/bin/curl -XPOST https://api.twilio.com/2010-04-01/Accounts/$accountsid/Calls.json --data-urlencode "To=$number" --data-urlencode "From=$callerid" --data-urlencode "Url=http://demo.twilio.com/docs/voice.xml" -u $accountsid:$authtoken
(the voice.xml demo is, in fact, Rick Astley)

Re: SSH Emergency Access

#42

Post by 8organicbits »

noodlesUK wrote: post #2

I think another thing we might want to learn about is how to sound the alarm when the break glass is used. Is there an easy way of doing that with SSH? Running a command to page the ops/security team when a server receives a login attempt with an emergency credential?

Login shell for emergency accounts could be a script that "sounds the alarm" and then drops to a bash shell.

Edit: ooo @rsync just gave another good approach

Re: SSH Emergency Access

#43

Post by harikb »

munchbunny wrote: post #40
alexandrerond wrote: post #35

I'm very confused, given Yubikeys have smart card fuctionality and they can be used by gpg-agent to SSH with the regular gpg key (you can add to authorized_keys just like any other keys) and you don't have to go through this whole mess of setup to create a CA and install it. What am I missing?

It's a chicken and egg problem: if you can't SSH into the machine, how do you add your key to the SSH config on the target machine? You could use a very long lived key, but then as soon as you have multiple people who might need production SSH access, you've got access control and revocation issues. The SSH CA is a good minimal solution, because the CA can issue only short-lived SSH keys (few hours at a time) that yo…

This can also work as a solution where the “setup” (of trusting CA) is baked in to the image. Then there is no ssh related setup until the day you actually need to ssh to the host. And you get the guarantee that no ssh login can happen until you issue a temp-pair.

This is actually quite useful for deploying clusters of machines that one doesn’t want normal ssh access until there is a real need. I think this was also mentioned in another comment

Re: SSH Emergency Access

#44

Post by lazyant »

benedikt wrote: post #36
lazyant wrote: post #26

Earlier quoted context omitted.

For people asking: you can create a resetfw.sh script, for iptables: #!/bin/bash iptables -P INPUT ACCEPT iptables -P FORWARD ACCEPT iptables -P OUTPUT ACCEPT iptables -t nat -F iptables -t mangle -F iptables -F iptables -X chmod +x resetfw.sh and add it for ex to /etc/cron.hourly directory This way you can test your iptables rules and they'll get clear at every hour. Once you check they are OK you can delete this cr…

https://manpages.debian.org/stretch/iptables/iptables-apply.... Or use `at` to run `iptables-restore`. Simpler than setting up a cronjob (and if youre doing it manually, cron has a bunch of gotchas that at least bite me in the ass once in a blue moon).

Yes. Although iirc (it may have changed, haven't looked "recently" the iptable- commands are distro specific, as in not all of them have / had them).

Re: SSH Emergency Access

#45

Post by raincom »

oars wrote: post #25
isatty wrote: post #15

Earlier quoted context omitted.

Yep, the most common way I've lost access to machines is by messing up the iptables/ipfw rules. Read a post here about avoiding that by having a timed reset with sleep.

This has happened to me as well. Where could I read about this method?

Maybe this:

   service network stop && sleep 10 && service network start

Re: SSH Emergency Access

#46

Post by lazyant »

taftster wrote: post #34
lazyant wrote: post #26

Earlier quoted context omitted.

For people asking: you can create a resetfw.sh script, for iptables: #!/bin/bash iptables -P INPUT ACCEPT iptables -P FORWARD ACCEPT iptables -P OUTPUT ACCEPT iptables -t nat -F iptables -t mangle -F iptables -F iptables -X chmod +x resetfw.sh and add it for ex to /etc/cron.hourly directory This way you can test your iptables rules and they'll get clear at every hour. Once you check they are OK you can delete this cr…

Or possibly just turn iptables off, in the same cron.hourly.

Ah yes, that's simpler: systemctl stop iptables. Also need to do systemctl disable iptables just in case, otherwise if the server reboots the iptables service will restart.

Re: SSH Emergency Access

#47

Post by dsr_ »

jon-wood wrote: post #24
marcosdumay wrote: post #21

Earlier quoted context omitted.

> All remote access was done using a short term (~1 day) ssh keys. There was an authentication service to generate these. This is weird. Really weird. Did that service use a more secure authentication storage than a password protected key?

It’s really not - by limiting the life of keys, and having a service generating them, you can more effectively lock things down when someone leaves, rather than going round revoking keys from servers. Something we’re experimenting with at work is AWS Instance Connect, which uses your AWS credentials to push a key to a target instance with 1 minute validity - no more managing keys on instances, and revoking access is…

As opposed to having a few bastion-hosts, and requiring people to log in there in order to then ssh on to their final destinations -- in that case, revoking their keys is as simple as wiping their accounts on the bastion hosts.

Re: SSH Emergency Access

#49

Post by CodeWriter23 »

tashian wrote: post #29
timeattack wrote: post #14

It's cool and interesting application of the technology, but doesn't really seem to be practical. When you're unable to access machine using your standard SSH keys usually it means that it's highly unlikely that it will be possible to login remotely via other means. As an emergency login there are two common options: * in case of cloud: use remote VM console provided by the hosting provider. * in case of bare-metal:…

Hey there — I'm the author of this post. There's a few scenarios where I imagined this approach being useful: * If you have any kind of remote dependency in your SSH auth flow (LDAP, or an online CA, or automated Ansible playbooks to push keys), any of those might fail and render the host otherwise inaccessible. * It's becoming more common to not ever SSH into machines. So, what if emergency SSH access is the only wa…

There’s also the access control feature of this approach. You can give someone temp access to a host.
Post reply on HN