SSH Emergency Access • Hacker News Board

Re: SSH Emergency Access

#71

Post by sidpatil » Sat, Jul 04, 2020, 11:40 PM UTC

timeattack wrote: post #14

It's cool and interesting application of the technology, but doesn't really seem to be practical. When you're unable to access machine using your standard SSH keys usually it means that it's highly unlikely that it will be possible to login remotely via other means. As an emergency login there are two common options: * in case of cloud: use remote VM console provided by the hosting provider. * in case of bare-metal:…

Not sure about other vendors, but I know Cisco offers dial-in capabilities for managing routers, switches, etc. The dial-in modem on the router is connected to a landline.

Has this approach ever been taken by server admins?

Re: SSH Emergency Access

#72

Post by peterwwillis » Sun, Jul 05, 2020, 1:22 AM UTC

tashian wrote: post #29

timeattack wrote: post #14

It's cool and interesting application of the technology, but doesn't really seem to be practical. When you're unable to access machine using your standard SSH keys usually it means that it's highly unlikely that it will be possible to login remotely via other means. As an emergency login there are two common options: * in case of cloud: use remote VM console provided by the hosting provider. * in case of bare-metal:…

Hey there — I'm the author of this post. There's a few scenarios where I imagined this approach being useful: * If you have any kind of remote dependency in your SSH auth flow (LDAP, or an online CA, or automated Ansible playbooks to push keys), any of those might fail and render the host otherwise inaccessible. * It's becoming more common to not ever SSH into machines. So, what if emergency SSH access is the only wa…

> While you could use the cloud VM console for emergency access in these cases, having a hardware key provides even more security and would let you turn off cloud VM access.

I'm not sure it's more secure, but I suppose it depends on the provider. Your control of your account's admin key (or password) is the last bastion of security for most providers.

> Of course if you broke your SSHD config, or have a network issue that prevents you from reaching the host, this won't magically fix any of that. IPMI is good for that though.

This is why I just use the providers' emergency management (or IPMI). Easier to have one method of emergency access that always works regardless of the guest. The guest's root (or emergency) account can still have a pretty darned complex password.

Re: SSH Emergency Access

#73

Post by bubblesorting » Mon, Jul 06, 2020, 4:47 AM UTC

tashian wrote: post #29

timeattack wrote: post #14

It's cool and interesting application of the technology, but doesn't really seem to be practical. When you're unable to access machine using your standard SSH keys usually it means that it's highly unlikely that it will be possible to login remotely via other means. As an emergency login there are two common options: * in case of cloud: use remote VM console provided by the hosting provider. * in case of bare-metal:…

Hey there — I'm the author of this post. There's a few scenarios where I imagined this approach being useful: * If you have any kind of remote dependency in your SSH auth flow (LDAP, or an online CA, or automated Ansible playbooks to push keys), any of those might fail and render the host otherwise inaccessible. * It's becoming more common to not ever SSH into machines. So, what if emergency SSH access is the only wa…

> It's becoming more common to not ever SSH into machines

This is a reality for me. At work we run a handful of distributed clusters, if anyone does an equivalent of sshing into a box and poking around (in our case, `kubectl exec`), the infrastructure team gets an alert, then follows up with whoever invoked the command. If they are doing debugging, we shift whatever resources they need into dev. If they are not debugging, they will probably get questioned by their boss. (fortunately, most of the time this chat results in, "oh wow I didn't know about the APM/Metrics/Graphs/Logs/etc setup we had, I'll check that next time)

Re: SSH Emergency Access

#74

Post by jon-wood » Mon, Jul 06, 2020, 12:28 PM UTC

dsr_ wrote: post #47

jon-wood wrote: post #24

Earlier quoted context omitted.

It’s really not - by limiting the life of keys, and having a service generating them, you can more effectively lock things down when someone leaves, rather than going round revoking keys from servers. Something we’re experimenting with at work is AWS Instance Connect, which uses your AWS credentials to push a key to a target instance with 1 minute validity - no more managing keys on instances, and revoking access is…

As opposed to having a few bastion-hosts, and requiring people to log in there in order to then ssh on to their final destinations -- in that case, revoking their keys is as simple as wiping their accounts on the bastion hosts.

Even with a few bastion hosts things get hard to track quickly as you end up with multiple clusters (dev/staging/UAT/production), and potentially multiple production clusters in different regions.