This reminds me of the ugliest hack I have ever written. For context: passwords rotate every 90 days, there are different passwords for client facing and "internal" servers, (and different passwords for linux machines vs Windows machines). All connections to client facing servers (which is my job) must go via: 1) a VPN 2) a "local" jumphost (both ssh/rdp), only accessible via VPN 3) a "remote" jumphost (also, both ss…
SSH hacks – a little sanity for remote workers
201–210 of 230 posts
Re: SSH hacks – a little sanity for remote workers
#202It’s a bit annoying that the word hack is used to mean tips and tricks here. As hacking ssh is big area
https://stallman.org/articles/on-hacking.html
However if someone said "hacking ssh" without the context of hacker news, I can see why the general computer person would probably think of the newer definition, which implies cracking or seeking security vulnerabilities.
Re: SSH hacks – a little sanity for remote workers
#203ssh not having support for --password argument is a big drawback. Usually requires to use some weird workarounds, especially when copying ssh keys to the machine are not an option.
I think sshpass might be a workaround, but sometimes you can just dig a little deeper into your situation and find if you can add authorized keys somehow.
I recall using a router with ssh access, but it didn't run true ssh with a .ssh/authorized_keys, but there turned out to be a way to add keys to the config for automated access.
Re: SSH hacks – a little sanity for remote workers
#204Earlier quoted context omitted.
It's quite likely that what you want in this case is to use SSH forced commands with sshkey auth and a remote account exclusively dedicated to serving this one request. https://binblog.info/2008/10/20/openssh-going-flexible-with-...
But that would be server-side, if I'm reading it right? I'm trying to remember the last time I needed password auth and wanted to do it in an automated fashion (like I said, it is rare), I think it was a router where the filesystem was read-only (only /var and /tmp writable, or something like that, so can't set authorized keys).
Forced commands are implemented on both sides of the session, as the previously linked reference ... doesn't entirely make clear. Locally you need to create, and generally configure, a specific key with the remote user@host Remotely, you associate that key with a specific command, in an authorized_keys file.
Re: SSH hacks – a little sanity for remote workers
#205Earlier quoted context omitted.
Is there much of an upside to doing the multiplexing on the client side instead of on the server (with something like tmux)? It seems to me like the session persistence tmux gives would be worthwhile if you have a lot of simultaneous sessions going.
scp and sftp will multiplex too. Remote tab-completion for file transfers also becomes much more approachable this way. scp remote:~/err[tab] .[enter] Also, bastion connections get multiplexed as well so connecting to multiple different hosts behind the bastion goes faster.
Re: SSH hacks – a little sanity for remote workers
#206Earlier quoted context omitted.
If a bad guy gets root on one server you're currently connected via ssh with, and forwarding said key via, they can also access (unless otherwise firewalled) all systems that ssh key grants them access to. See also: https://github.com/FiloSottile/whoami.filippo.io Using a private key per system or set of systems means that such an attacker would only gain access to what that key grants them access to. My "github" key…
The SSH-agent forwarding model is the problem, I think; Every couple of years someone finds another semi-common scenario where it becomes vulnerable. I have, since 2010, practiced "jumps" only - that is, if I need to connect to a non-routeable host through a routeable one ("bastion host" or "jump host"), then the intermediate host is only used as a transport, by way of -L/-R forwarding of the final destination's ssh…
Re: SSH hacks – a little sanity for remote workers
#207Earlier quoted context omitted.
I think the proper solution would be 1) don't use forwarding unless necessary 2) setup touch-to-use on a Yubikey so even with your scenario the attacker couldn't just use the key (it requires touch to be used).
I've (admittedly, conceptual only) a couple problem with this approach I'd love to hear folks' thoughts about, with regards to "touch-to-use". 1. I still have a need to "for s in $list_of_servers; ssh -A $s '..'; done" How many times do I have to push the button? Is there a setting that allows that "push to allow" for a short while? Great, if so: 2) when I ssh into a system and touch the button to allow it, and the p…
Re: SSH hacks – a little sanity for remote workers
#208- Customise ~/.ssh/config to suite your needs (be careful with storm - manage ssh like a boss, it helps when scripting or searching hosts but has a outstanding bug converting keywords to lowercase [1])
- Use ed25519 key over RSA
- OpenSSH 8.1 added support for FIDO/U2F (use your YubiKey or equivalent)
- Put `IPQoS lowdelay throughput` in your ~/.ssh/config if you run a rolling release (e.g. Arch, Gentoo) or your openssh rolls via homebrew on macOS. latest openssh client with older version of sshd may produce weird disconnection issues (server reset connection, client side is able to connect but terminal hangs in 5~10s). Spend quite some time digging only to find that it was caused by default change for IPQoS (to IPQoS af21 cs1) introduced in OpenSSH 7.8p1 [3]
- leverage ssh-copy-id
- ssh -vvv | ssh -G (troubleshooting from client side)
- /usr/sbin/sshd -p 2222 -f /path/to/sshd_config -D -ddd (troubleshooting sshd server side)
- Be careful with `UsePAM no`, make sur
- use AllowUsers / DenyUsers vs DenyGroups vs AllowGroups , mind the order
- know how to use ssh-add / ssh-keygen / ssh-agent / ssh-keyscan
- audit SSH config (ssh-audit / lynis), version control ssh_config / sshd_config properly if possible
- openssh + tmux ;-)
Personal favourite tips/tricks:
- ssh -D (used to use this dynamic port forwarding, open a local Socks5 proxy to punch hole in firewall, encrypt traffic, it worked for a while against the infamous GFW, only a little while though)
- ssh -L | -R TCP forwarding
- ssh -X | -Y X11 forwarding (run X11 apps remotely and display it on X Server locally)
- More personal SSH tricks put together over the years, surprise to find that my person OpenSSH notes are 150+ pages in Google Docs, sorry can't put all in a comment... [2]
- RTFM works, OpenSSH is worth the time ;-)
[1]: https://github.com/emre/storm/issues/157
[2]: https://sites.google.com/site/imterry/computer/tools/ssh
[3]: ssh(1)/sshd(8): the default IPQoS used by ssh/sshd has changed. https://www.openssh.com/releasenotes.html
Re: SSH hacks – a little sanity for remote workers
#209I am missing certificates on this list, you never have to distribute individual public keys. And you can sign certificates with an API. It is also nice to give someone temporary access, because they are valid in a time interval.
[1] https://smallstep.com/blog/use-ssh-certificates/
Re: SSH hacks – a little sanity for remote workers
#210Try using ssh-import-id to fetch your (or a friend or colleague's) public key from Github!
Depending on use case, though, this can be a bit sketch. At smallstep we like SSH certificates, which make life similarly easy on everyone with a bunch of other benefits. You can find a couple relevant posts on our blog if you’re interested.
Incidentally, GitHub now supports SSH certificates (for enterprise edition, at least).