What makes you think that the accounts were not compromised only recently?
Timeline of the xz open source attack
71–80 of 482 posts
Re: Timeline of the xz open source attack
#72I wonder, once the attacker gained commit permissions, were they able to rewrite and force push existing commits? In that case rolling back to older commits may not be a solution. If my speculation is correct then the the exact date on which access was granted must then first be known, after that a trusted backup of the repo from before that date is needed. Ideally Lasse Collin would have a daily backup of the repo.…
Force pushes tend to be noticed easily. All it takes is for one external developer to try to pull to see the failure. And it's actually hard to do because you need to comb through the tree to update all the tags that point to the old commits. On top of that it obviously breaks any external references to the commit IDs (e.g. in distro or build configurations), all the way up to cryptographic signatures that might have been made on releases.
I think it's a pretty reasonable assumption that this didn't happen, though it would be nice to see a testimony to that effect from someone trustworthy (e.g. "I restored a xz checkout from a backup taken before 5.6.0 and all the commit IDs match").
Re: Timeline of the xz open source attack
#73Earlier quoted context omitted.
The Jigar Kumar nudges are so incredibly rude. I would have banned the account, but perhaps they contributed something positive as well that isn't mentioned. I wonder if it would be possible to crowdsource FOSS mailing list moderation.
There is a good chance that everyone in that thread except the original maintainer is in on the act. It's likely that all those accounts are managed by a single person or group. Targeting just one account for rudeness isn't going to help, if that's true.
The mechanism employed here seems like the good cop, bad cop interrogation/negotiation technique. There is the one person who has taken care to show cultural and mission alignment. Then there are several misaligned actors applying pressure which the first person can relieve.
How to identify and defuse: https://www.pon.harvard.edu/daily/batna/the-good-cop-bad-cop...
Re: Timeline of the xz open source attack
#74What makes you think that the accounts were not compromised only recently?
Not OP, but the fact that they only appear in this context makes me believe that their are specifically set up for this task. No regular person has that good opsec just to push patches to a random library.
Re: Timeline of the xz open source attack
#75A "good" side effect of this for OSS maintainers going on, is that now any time an entitled user starts being too pushy or too... well, entitled , they can be given a canned response: Are you trying to pull an xz attack on me?
Ah, the old "undercover cops can't lie about not being a cop, just ask them" technique.
Re: Timeline of the xz open source attack
#76This seems very difficult to defend against. What is a project with a single burnt-out committer to do?
It's not a perfect solution, but I think Big Companies could have a role to play here. Some kind of tenure/patronage system, where for every $1B a Big Company makes in profit, they employ one critical open source maintainer with a decent salary (like $200k or something). The job would only have two requirements: 1) don't make everyone so mad that everyone on the Internet asks for you to be fired, and 2) name a suitable replacement when you're ready to move on. The replacement would become an employee at Big Company, which means Big Company would need to do whatever vetting they normally do (background checks, real address to send paychecks and taxes, etc).
In this scenario, Jia Tan would not be a suitable replacement, since they don't friggin' exist.
Yes, there's problems with this approach. When money gets involved, incentives can get distorted. It limits the pool of acceptable maintainers to those employable by a Big Company. But I think these are solvable problems, especially if there's a strong culture of maintainer independence. It provides a real improvement over the current situation of putting so much pressure on individuals doing good work for no benefit.
Re: Timeline of the xz open source attack
#77Is the real identity of Jia Tan known, even by Lasse Collin? I would think a "real identity" should be required by linux distros for all /major/ open source projects/library committers which are included in the distro, so that we can hold folks legally accountable
Open source fundamentally does not work that way. There are many important open source contributors who work pseudonymously. Google's Know, Prevent, Fix blog post floated the idea of stronger identity for open source in https://security.googleblog.com/2021/02/know-prevent-fix-fra... and there was very significant pushback. We learned a lot from that. The fundamental problem with stronger identity is that spy agencies…
The blog post clarified it's about maintainers of critical packages, not all contributors. This could be limited to packages with just one or two maintainers, especially newer ones. And they could remain somewhat anonymous, providing their information to trusted third parties only. If some maintainers don’t accept even this, their commits could be put into some special queue that requires additional people to sign off on them before they get accepted downstream. It's not a complete fix, but it should help.
Re: Timeline of the xz open source attack
#78"It's also good to keep in mind that this is an unpaid hobby project." ~ Lasse Collin, 2022-06-08.
As someone working in security, the fact that _foundational_ pieces of the computing/networking rely on motivated individuals and essentially goodwill is mind blowing.
There are great aspects to the FOSS movement, but the risks – particularly the social engineering aspects as demonstrated here – and potential blast radius of supply chains like this... We take it all for granted and that is lining up to bite us hard, as an industry.
Re: Timeline of the xz open source attack
#79I wonder if netflix will make a movie from this story, if you read the timeline it really sounds like a well written thriller.
It’s not compelling enough unless we find out who was behind it, which is probably unlikely.
Re: Timeline of the xz open source attack
#80The social side of this is really haunting me over the last days. It's surprisingly easy to pressure people to giving up control. I've been there myself. I can't even imagine how devastating this must be to the original author of XZ, especially if he is dealing with other personal issues as well. I hope at least this will serve a strong example to other open source people, to never allow others to pressure them into…
The Jigar Kumar nudges are so incredibly rude. I would have banned the account, but perhaps they contributed something positive as well that isn't mentioned. I wonder if it would be possible to crowdsource FOSS mailing list moderation.
The xz list traffic was remarkably low. More than a few times over the years, I thought it broke or I was unsubscribed.
Messages like Jigar’s are kind of par for the course.