Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Managing SSH Access to AWS EC2 Instances Using SSM

github.com

Post reply on HN
Print view View original

21–30 of 37 posts

Re: Show HN: Managing SSH Access to AWS EC2 Instances Using SSM

#21

Post by marcinzm »

klohto wrote: post #4

Solving the SSH over SSM is sadly useless right now. The whole point of why we began using SSM was logging. 1) We soon discovered that using bash (and not sh that SSM signs you into) doesn’t format the logs correctly and makes them unreadable 2) Using SSH over SSM doesn’t log at all Both of these problems are on AWS github but as always, they don’t respond.

Does SSM generate unique users for each login or does it use a generic user? The later would make any logging outside of SSM difficult since there's no way to tie it to a specific IAM user except time windows.

Re: Show HN: Managing SSH Access to AWS EC2 Instances Using SSM

#22

Post by ses1984 »

leg100 wrote: post #19
ses1984 wrote: post #17

Earlier quoted context omitted.

I'm pretty sure this only works if you use images with Google's sauce added, which means your instances run Google daemons that do things like network, user, and ssh config.

Correct albeit no Google daemons, just OpenSSH server, PAM and NSS configurations: https://cloud.google.com/compute/docs/oslogin/#how_os_login_...

That's good to know.

...But if you use an image with Google's sauce, there are Google daemons running on it, just not for this.

https://cloud.google.com/compute/docs/images/install-guest-e...

Re: Show HN: Managing SSH Access to AWS EC2 Instances Using SSM

#23

Post by marcinzm »

vegardx wrote: post #7

You can also use ec2-instance-connect[0] to enable users to generate ephemeral keys that are valid for a few minutes, authenticated with IAM. The benefit is that it's all native after you've pushed your keys using either aws-cli or mssh. It has more or less the same benefits that SSM has, and you can use the same method with ProxyCommand to establish authentication before connecting. You can also chain it using Proxy…

Does ec2-instance-connect use the generic system account or does each AIM user/role get it's own instance account? Asking from an auditing and logging perspective, as in is there a way to tie back a command which was run back to an individual human directly?

Re: Show HN: Managing SSH Access to AWS EC2 Instances Using SSM

#24

Post by klohto »

marcinzm wrote: post #21
klohto wrote: post #4

Solving the SSH over SSM is sadly useless right now. The whole point of why we began using SSM was logging. 1) We soon discovered that using bash (and not sh that SSM signs you into) doesn’t format the logs correctly and makes them unreadable 2) Using SSH over SSM doesn’t log at all Both of these problems are on AWS github but as always, they don’t respond.

Does SSM generate unique users for each login or does it use a generic user? The later would make any logging outside of SSM difficult since there's no way to tie it to a specific IAM user except time windows.

You can specify the behavior. There is no unique user for each login, by default you are dropped to generic ssm-user. It's possible to set up a log-in into specific user based on the IAM username.

> The later would make any logging outside of SSM difficult There is a way to tie IAM user to SSM. You have to request SSM permissions and AWS logs that. AWS logs pretty much every single API call so it's just matter of storing and auditing correctly.

Re: Show HN: Managing SSH Access to AWS EC2 Instances Using SSM

#26

Post by marcinzm »

klohto wrote: post #24
marcinzm wrote: post #21

Earlier quoted context omitted.

Does SSM generate unique users for each login or does it use a generic user? The later would make any logging outside of SSM difficult since there's no way to tie it to a specific IAM user except time windows.

You can specify the behavior. There is no unique user for each login, by default you are dropped to generic ssm-user. It's possible to set up a log-in into specific user based on the IAM username. > The later would make any logging outside of SSM difficult There is a way to tie IAM user to SSM. You have to request SSM permissions and AWS logs that. AWS logs pretty much every single API call so it's just matter of sto…

>There is a way to tie IAM user to SSM. You have to request SSM permissions and AWS logs that. AWS logs pretty much every single API call so it's just matter of storing and auditing correctly.

If you're using a generic ssm-user (and going to ssh so there's no ssm logs of the commands) then wouldn't you not be able to distinguish the commands (say in auditd) of two people logging in at the same time?

I'm reminded of RDS which does not store enough logs anywhere to reconstruct which IAM user/role was running specific commands when using IAM Authentication unless you create individual users in the DB.

Re: Show HN: Managing SSH Access to AWS EC2 Instances Using SSM

#27

Post by vegardx »

marcinzm wrote: post #23
vegardx wrote: post #7

You can also use ec2-instance-connect[0] to enable users to generate ephemeral keys that are valid for a few minutes, authenticated with IAM. The benefit is that it's all native after you've pushed your keys using either aws-cli or mssh. It has more or less the same benefits that SSM has, and you can use the same method with ProxyCommand to establish authentication before connecting. You can also chain it using Proxy…

Does ec2-instance-connect use the generic system account or does each AIM user/role get it's own instance account? Asking from an auditing and logging perspective, as in is there a way to tie back a command which was run back to an individual human directly?

It has to use an existing system account, yes. You do lose a lot of auditing abilities this way, since you cannot capture the content. For me this is more about establishing a secure way to grant people access to instances, without something like LDAP or SSH CA.

LDAP sends passwords in cleartext over an encrypted channel, I don't like the idea of someone else on the host I'm connecting to being able to read my password. Naturally you can use key based authentication with it.

I know that some people are required by law or regulation to log all these things, but personally I don't see much benefit of such audit logs. All bets are off when someone has local access to a machine. And there's this whole debate about using MitM-proxies for logging and security and the potential security implications they pose.

All that said, step-ca[0] and vault[1] looks really promising to provide some kind of provider independent way to authenticate in a secure manner. You also have teleport[2] but I haven't looked much into it.

[0] https://smallstep.com/docs/cli/ssh/ [1] https://www.vaultproject.io/docs/secrets/ssh/signed-ssh-cert... [2] https://gravitational.com/teleport/

Re: Show HN: Managing SSH Access to AWS EC2 Instances Using SSM

#28

Post by klohto »

marcinzm wrote: post #26
klohto wrote: post #24

Earlier quoted context omitted.

You can specify the behavior. There is no unique user for each login, by default you are dropped to generic ssm-user. It's possible to set up a log-in into specific user based on the IAM username. > The later would make any logging outside of SSM difficult There is a way to tie IAM user to SSM. You have to request SSM permissions and AWS logs that. AWS logs pretty much every single API call so it's just matter of sto…

>There is a way to tie IAM user to SSM. You have to request SSM permissions and AWS logs that. AWS logs pretty much every single API call so it's just matter of storing and auditing correctly. If you're using a generic ssm-user (and going to ssh so there's no ssm logs of the commands) then wouldn't you not be able to distinguish the commands (say in auditd) of two people logging in at the same time? I'm reminded of R…

> If you're using a generic ssm-user (and going to ssh so there's no ssm logs of the commands) then wouldn't you not be able to distinguish the commands (say in auditd) of two people logging in at the same time?

But that requires auditing directly on EC2, right? (You can indeed sync the logs somewhere, but that diminished the beauty of out of box logging)

> I'm reminded of RDS which does not store enough logs

AFAIK, it's not possible with their current implementation to accomplish this. We just swallowed that it won't work and for the sake of security created seperate users.

Re: Show HN: Managing SSH Access to AWS EC2 Instances Using SSM

#29

Post by marcinzm »

vegardx wrote: post #27
marcinzm wrote: post #23

Earlier quoted context omitted.

Does ec2-instance-connect use the generic system account or does each AIM user/role get it's own instance account? Asking from an auditing and logging perspective, as in is there a way to tie back a command which was run back to an individual human directly?

It has to use an existing system account, yes. You do lose a lot of auditing abilities this way, since you cannot capture the content. For me this is more about establishing a secure way to grant people access to instances, without something like LDAP or SSH CA. LDAP sends passwords in cleartext over an encrypted channel, I don't like the idea of someone else on the host I'm connecting to being able to read my passwo…

Thanks for the links, Teleport seems especially interesting.

Yeah, we are required by contractual obligations to log these things or at least sudo commands. Don't disagree that it's less than useful but only so many things you can argue about. The logging we can do with auditd but we need unique instance users for that to be useful.

Re: Show HN: Managing SSH Access to AWS EC2 Instances Using SSM

#30

Post by vegardx »

williamdclt wrote: post #16

Very funny, we received a ticket from a security audit a week ago asking us to get rid of our bastion and use SSM instead. Having never heard of SSM, being in a rush to a deadline and lacking skills with ops, we kinda panicked. 10min later, we had found the SSM interface in the AWS console, found it extremely easy to use and had deleted our bastion :) Note: the README says "I found myself in a situation where I could…

You can always just pipe things over ssh. And now that ssm supports local port forwarding you can easily use netcat to send files back and forth.
Post reply on HN