Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reflections on Distrusting xz

joeyh.name

Post reply on HN
Print view View original

221–230 of 335 posts

Re: Reflections on Distrusting xz

#221

Post by dralley »

liendolucas wrote: post #205
dralley wrote: post #194

Earlier quoted context omitted.

There is no reason to assume that. Even if you assume every commit since Jia became a maintainer is malicious, the version from 3 years ago is perfectly fine. Zstd has a number of benefits over Xz that may warrant its use as a replacement of the latter, and this will likely be a motivating factor to do so. But calling it entirely rotten is going way too far IMO

What keeps ringing in my head is the "." that was found that invalidates compilation. I personally don't buy it (but is my opinion).

What do you mean "don't buy it"?

Re: Reflections on Distrusting xz

#222

Post by sylware »

cletus wrote: post #170

The US spends $30-60 billion a year on agriculture and subisides [1]. This is controversial for all the obvious reasons. Without subsidies we'd waste less food but overproduction of food is an intentional objective of these programs. Why? Because if there's a major drought or crops are lost to ice or snow or flooding, Americans won't starve. It's why we have things like the US government having a reserve of over a bi…

It may be paying for minimal (including the SDK), but able to do a good enough job, ultra stable in time reference software/network protocol/file format maintainance.

It excludes nearly all software out there (even open source, and closed source are de-facto excluded), because most "developers" are only a bunch of scammers heavy on planned obsolescence.

That includes software "maintained" by the academic sector, like your have with MIT media labs and the nice "donations" from bill gates (to probably steer it the way he wanted, I would not be surprised this is not alien to c++ in gcc... one of the biggest mistakes in open source software), that revealed in the epstein files.

To say the least, it is far from ez. If you are an _honest_ dev, you know it is extrutiatingly hard to justify a permanent income.

Re: Reflections on Distrusting xz

#223

Post by StayTrue »

belorn wrote: post #130
prmoustache wrote: post #78

Earlier quoted context omitted.

I wouldn't limit that to OSS projects. How many of them managed to get hired and are working for Microsoft, Apple, Google, Oracle or Amazon? In some cases they don't even need to introduce backdoors themselves but just review and spot bugs they don't correct or raise issues for but communicate to mothership. They could even work in team with having one building the backdoor and the other approving the code. Most comp…

Most of the time they don't need infiltrators. Governments can just pressure companies with export controls or warrantless surveillance to get backdoors into commercial systems. OSS projects require different methods because the more direct method would be discarded by the community and forked.

> Most of the time they don't need infiltrators. Governments can just pressure companies with export controls or warrantless surveillance to get backdoors into commercial systems.

Or they simply pay companies with a “support contract” in return for embedding spyware that sells out customers. Seen that first hand (private key exfiltration), resigned the same day.

Lots of comments saying we need to do something about the OSS supply chain but in my estimation the problem is much worse with closed source commercial software.

Re: Reflections on Distrusting xz

#224

Post by acdha »

Gormo wrote: post #203
cletus wrote: post #170

The US spends $30-60 billion a year on agriculture and subisides [1]. This is controversial for all the obvious reasons. Without subsidies we'd waste less food but overproduction of food is an intentional objective of these programs. Why? Because if there's a major drought or crops are lost to ice or snow or flooding, Americans won't starve. It's why we have things like the US government having a reserve of over a bi…

It's not valid to regard the potential of negative consequences in any sphere of human life as a "national security risk". It's not valid to presume that the only way to mitigate risks is through top-down political intervention. And it's absolutely not valid to presume that making FOSS communities dependent on political subsidies would not have much worse and much longer term consequences than the problems we are try…

Your argument falls apart when you remember that the U.S. federal, state, and local governments are critically dependent on open source software – not just directly in things like Linux servers or Chrome/Edge/Firefox but also open source components used in appliances or compiled into commercial software. It is quite reasonable to argue that even a narrow approach of improving only components they run would be justifiable on those grounds and it’d be a tiny part of, say, NIST’s budget to fund developers directly or to pay some group like the Linux or Apache foundations to support an open source maintainership team.

Re: Reflections on Distrusting xz

#225

Post by tripflag »

dralley wrote: post #194
liendolucas wrote: post #176

Earlier quoted context omitted.

I think at this point is clear that everybody has to assume that XZ is completely rotten and can no longer be trusted. Is it XZ easy to replace with some other compression tool? Or has it been so widely adopted that is going to take huge effort moving out of it?

There is no reason to assume that. Even if you assume every commit since Jia became a maintainer is malicious, the version from 3 years ago is perfectly fine. Zstd has a number of benefits over Xz that may warrant its use as a replacement of the latter, and this will likely be a motivating factor to do so. But calling it entirely rotten is going way too far IMO

I always use "zstd --long=31 -T0 -19" to compress disk images, since that is a usecase where it generally offers vastly superior compression to xz, deduplicating across bigger distances.

XZ offers slightly better compression on average, but decompression is far slower than Zstd.

Re: Reflections on Distrusting xz

#226

Post by gammalost »

funcDropShadow wrote: post #18
gammalost wrote: post #13

Earlier quoted context omitted.

You can't really remove dependencies in open source. It is so intertwined at this point that doing it would be too expensive for most companies. I think the solution is to containerize, containerize and then containerize some more times and make it all with zero trust in mind.

Docker containers are not really a security measure.

It is a security measure. Sure it doesn't secure anything in the container itself. But it secures the container from other containers. Code can (as proven) not be trusted, but the area of effect can be reduced.

Re: Reflections on Distrusting xz

#227

Post by mmd45 »

dralley wrote: post #194
liendolucas wrote: post #176

Earlier quoted context omitted.

I think at this point is clear that everybody has to assume that XZ is completely rotten and can no longer be trusted. Is it XZ easy to replace with some other compression tool? Or has it been so widely adopted that is going to take huge effort moving out of it?

There is no reason to assume that. Even if you assume every commit since Jia became a maintainer is malicious, the version from 3 years ago is perfectly fine. Zstd has a number of benefits over Xz that may warrant its use as a replacement of the latter, and this will likely be a motivating factor to do so. But calling it entirely rotten is going way too far IMO

There is an interesting argument to be made that pre-JT xz code is probably pretty secure due to the fact that the threat actors would have already audited the code for existing exploits prior to exerting effort to subvert it.

Re: Reflections on Distrusting xz

#228

Post by FdbkHb »

adql wrote: post #104
j16sdiz wrote: post #33

Earlier quoted context omitted.

... but `xz` is pretty much feature complete to me. Lasse Collin was doing bug-fix-only release just fine.

The start of the attack was few fake accounts trying to shame the maintainer for not "developing" it constantly and so give maintainer rights to someone else. And there wasn't really anyone to say "nope, it's fine, fuck off"

> And there wasn't really anyone to say "nope, it's fine, fuck off"

That's because for people for whom the project is doing fine and they haven't experienced any bug, why would they go to the mailing list, forum or whatever other form of communication channel the project has?

One has to understand and keep in mind that places that can gather feedback will invariably attract more of the negative kind than the positive kind because people who are happy are not motivated to say they're happy. Those people wouldn't even know people were complaining about xz.

There's thousands of libraries/independent software projects installed on any computer. No one has the time to check the place of all those software projects and go there just to say "hey, I'm happy, no need to change anything, thanks?", right?

People who are discontent with something on the other hand are sure to be vocal about it. But just because they're the most vocal doesn't mean they are the majority of your users.

Re: Reflections on Distrusting xz

#229

Post by ghaff »

tomn wrote: post #212
ghaff wrote: post #161

Earlier quoted context omitted.

I'm not sure why the downvotes. That seems to be a statement of fact. You can do a certain amount of identity obfuscation online but for anyone with a real professional profile you're generally not really anonymous if anyone really cares to find out your true name.

Me neither, i even provided a source, and it's easy to find other examples. There certainly are projects that allow anonymous contributions, but i doubt it's the majority of projects that one would consider important. For these kinds of projects you could make up an identity relatively easily and nobody would know, but you're screwing over the project (as they may need to remove your contributions if they find out),…

There are people who downvote things that present facts that aren't in accordance with how they think the world should be.

I do think it's difficult to verify identity in any reasonably acceptable lightweight way. That said, for the larger projects I'm most familiar with, a lot of people work for companies, attend conferences, etc. They may go by nicknames day to day, but they have known real identities and their professional existence wouldn't be possible without one.

Re: Reflections on Distrusting xz

#230

Post by dralley »

tripflag wrote: post #225
dralley wrote: post #194

Earlier quoted context omitted.

There is no reason to assume that. Even if you assume every commit since Jia became a maintainer is malicious, the version from 3 years ago is perfectly fine. Zstd has a number of benefits over Xz that may warrant its use as a replacement of the latter, and this will likely be a motivating factor to do so. But calling it entirely rotten is going way too far IMO

I always use "zstd --long=31 -T0 -19" to compress disk images, since that is a usecase where it generally offers vastly superior compression to xz, deduplicating across bigger distances. XZ offers slightly better compression on average, but decompression is far slower than Zstd.

IIRC memory consumption is generally worse for Zstd at comparable levels of compression. Which, these days, is generally fine, but my point is you can't thoughtlessly substitute the two.
Post reply on HN