Can anyone explain the Jenkins vulnerability that was used to initially gain access? Reading the CVEs didn't give me the impression that they enabled remote exploits
What can we learn from the matrix.org compromise?
21–30 of 75 posts
Re: What can we learn from the matrix.org compromise?
#22Earlier quoted context omitted.
Okay, I'll bite. What are you calling a dark pattern in assorted agents? Especially given that dark pattern implies intent to harm. (And I say this as someone looking at using an agent: If there's a gotcha, I'd like to know about it)
1) SSH agent will cache your passphrase. While that's the whole purpose of SSH agent, remember that nothing is more insecure than an unlocked secret. 2) SSH agent often starts automatically, frequently without user interaction (even if you specify `-i keyfile`). SSH client and DBus are both culprits here; there's also other culprits too. 3) There are often multiple different agents installed on Linux desktop systems.…
> 1) SSH agent will cache your passphrase. While that's the whole purpose of SSH agent, remember that nothing is more insecure than an unlocked secret.
There's one thing more insecure than an unlocked secret: a "secret" sitting in plain text on the filesystem.
Which is a common outcome if you advise people against using an agent and they don't share your opsec priorities.
Re: What can we learn from the matrix.org compromise?
#23Earlier quoted context omitted.
As in, no human should run a package manager in prod? (But salt/ansible/etc. running it is fine) Same idea as "if you're SSHing to prod, something is wrong" (where provisioning tools make all changes, logs are all aggregated and delivered in their own tool, and even debugging is built into the app or logging system).
More as, you should not modify images running in production. By human or machine. Rather build new images and roll over the fleet. If you need to debug, remove from production (quarantine) and work on it there. Don't run master / agent setups for ansible / salt anymore. You can still use them for creating images, which are later turned into running VMs. Think about it like containers. Do you update the contents of yo…
But golden images on Linux are, well, messy. It's very annoying to make a clean VM template without some post-provisioning like cloud-init. And for most shops if you're running cloud-init you could do that post-provisioning with Ansible or Salt. And since your images are built with Ansible/Salt in the first place you might as well just build each VM fresh and use the vendor's ISO. One less thing to maintain and update.
Plus when you're in a pinch, which never happens of course, you can make changes without having to roll your VMs.
I feel like Atomic distributions are basically the happy medium between the two worlds.
Re: What can we learn from the matrix.org compromise?
#24Earlier quoted context omitted.
More as, you should not modify images running in production. By human or machine. Rather build new images and roll over the fleet. If you need to debug, remove from production (quarantine) and work on it there. Don't run master / agent setups for ansible / salt anymore. You can still use them for creating images, which are later turned into running VMs. Think about it like containers. Do you update the contents of yo…
But golden images on Linux are, well, messy. It's very annoying to make a clean VM template without some post-provisioning like cloud-init. And for most shops if you're running cloud-init you could do that post-provisioning with Ansible or Salt. And since your images are built with Ansible/Salt in the first place you might as well just build each VM fresh and use the vendor's ISO. One less thing to maintain and updat…
It's a trade-off. The point is the matrix security lapse turned worse because they ran this master / agent setup. You can still use ansible (or similar), just do it localhost during the build process.
Yea it's easier to not do these things, because good security posture takes work to set up. Once you're on the immutable train, you'll find it's not actually harder day to day. You learn to deal with issues in the pinch another way.
On the point of building VMs fresh each time vs building golden images, you'll find you boot time reduced, your roll over more reliable and autoscaling more responsive. Why build the same thing dozens or hundreds of times? What happens if a remote package is updated in the middle of your upgrade? Does this sound messier to you?
Re: What can we learn from the matrix.org compromise?
#25Why is it considered safer to expose a VPN to the internet than SSH? Is it just that there is one exposed service for the organisation rather than one per machine?
Re: What can we learn from the matrix.org compromise?
#26Why is it considered safer to expose a VPN to the internet than SSH? Is it just that there is one exposed service for the organisation rather than one per machine?
You can expose one SSH box per organization (a “bastion”) and deploy SSH configs to clients that make it look like you have direct access to the hosts behind it.
Re: What can we learn from the matrix.org compromise?
#27Why is it considered safer to expose a VPN to the internet than SSH? Is it just that there is one exposed service for the organisation rather than one per machine?
SSH tunneling is handy but if you want to push anything else over it, its a pain for the "layperson". You're not going to have a great time supporting people with it. I've done it, it sucks. Scripts and special SSH config files are the pits. VPNs are way easier, they can support multiple access levels and roles, are often not blocked by other people's packet filters and firewalls and the good ones can even validate that a host is in "compliance" before they're allowed onto the network.
Re: What can we learn from the matrix.org compromise?
#28This is such a poorly written article:
* no detailed analysis of how the attack was undertaken. Its not even clear how the attacker managed to get in (was it a publicly exposed Jenkins? vulnerable bastion? what?)
* no analysis of what the existing matrix.org security perimeter looked like or how it could be made better.
* repetition of security tropes. Use VPN. Use Github Enterprise (wait wtf? Why not private repos in Github?). Don't use Ansible, use salt.
Ridiculous. I was looking forward to a nice long read about how this breach was undertaken. Hugely disappointed.
Re: What can we learn from the matrix.org compromise?
#29I have gone on some long verbal rants about the dark patterns (bordering on malicious behavior) exhibited by key agents such as SSH agent, GPG agent, Pageant, and the like. What can you learn from the compromise? Never use an agent. Kill it with fire^H^H^H^H -9.
How about using hardware tokens instead? With a right setup, private keys never leave it.
Hardware tokens are great but most people don't know how to use them, so they don't.
Re: What can we learn from the matrix.org compromise?
#30I have gone on some long verbal rants about the dark patterns (bordering on malicious behavior) exhibited by key agents such as SSH agent, GPG agent, Pageant, and the like. What can you learn from the compromise? Never use an agent. Kill it with fire^H^H^H^H -9.
How about using hardware tokens instead? With a right setup, private keys never leave it.
Hardware tokens are pretty alright until you need to use GPG-agent to enable their use.