Backdoor in upstream xz/liblzma leading to SSH server compromise

I got curious and decided to run 'ent' https://www.fourmilab.ch/random/ to see how likely the data in the bad stream was to be random. I used some python to split the data into 3 streams, since it's supposed to be the middle one that's "bad":

I used this regex to split in python, and wrote to "tmp":

    re.split(b'\xfd7zXZ', x)

    $ ent tmp2 # bad file payload
    Entropy = 4.157806 bits per byte.
    
    Optimum compression would reduce the size
    of this 48 byte file by 48 percent.
    
    Chi square distribution for 48 samples is 1114.67, and randomly
    would exceed this value less than 0.01 percent of the times.
    
    Arithmetic mean value of data bytes is 51.4167 (127.5 = random).
    Monte Carlo value for Pi is 4.000000000 (error 27.32 percent).
    Serial correlation coefficient is 0.258711 (totally uncorrelated = 0.0).
    
    $ ent tmp3 # urandom
    Entropy = 5.376629 bits per byte.
    
    Optimum compression would reduce the size
    of this 48 byte file by 32 percent.
    
    Chi square distribution for 48 samples is 261.33, and randomly
    would exceed this value 37.92 percent of the times.
    
    Arithmetic mean value of data bytes is 127.8125 (127.5 = random).
    Monte Carlo value for Pi is 3.500000000 (error 11.41 percent).
    Serial correlation coefficient is -0.067038 (totally uncorrelated = 0.0).
    

Anyone have a link to the git history? I guess we can use the ubuntu tarball for the evil version.

I've been learning NixOS for a few years now, and it would have been impossible without systemd. It's one heck of a learning curve, but when you get to the other side, you know something of great power and value. Certain kinds of complexity adds 'land' (eg. systemd) that can become 'real estate' (eg. NixOS), which in turn hopes to become 'land' for the next innovation, and so forth.

Whether this happens or not (whether it's the right kind of complexity) is really hard to assess up-front, and probably impossible without knowing the complex new technology in question very well. (And by then you have the bias of depending, in part, yourself on the success of the new tech, as you've committed significant resources to mastering it, so good luck on convincing skeptical newcomers!)

It's almost like a sort of event horizon -- once you know a complex new technology well enough to see whether or not it's useful, the conflict-of-interest makes your opinion unreliable to outsiders!

Nevertheless, the assessment process itself, while difficult to get right, is worth getting better at.

It's easy for impatience and the sensation of what I've taken to calling 'daunt' -- that intrinsic recoil that the mind has from absorbing a large amounts of information whose use case is not immediately relevant -- to dissuade one from exploring. But then, one never discovers new 'land', and one never builds new real estate!

[ Aside: This is why I'm a little skeptical of the current rebellion against frontend frameworks. Certainly some of them, like tailwind, are clearly adding fetters to an otherwise powerful browser stack. But others, like Svelte, and to some extent, even React, bring significant benefits.

The rebellion has this vibe like, well, users _should_ prefer more simply-built interfaces, and if they don't, well, they just have bad taste. What would be more humble would be to let the marketplace (e.g. consumers) decide what is preferable, and then build that. ]