Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Excellent succinct breakdown of the xz mess, from an OpenBSD developer

marc.info

Post reply on HN
Print view View original

11–20 of 54 posts

Re: Excellent succinct breakdown of the xz mess, from an OpenBSD developer

#11

Post by dpkirchner »

AwaAwa wrote: post #10
brenoguim wrote: post #9

> It does not plausibly pass for a typo because no typical editing glitch will leave a '.' character there. I would certainly attribute that to a typo if I was reviewing the code.

> The syntax error is a single period '.' as the first character on an otherwise empty line of C code. I believe the point is that given this context, it could/should not be construed as a typo.

If you received this patch from someone you trusted, and noticed the period, would you call it a typo or would you assume it is malicious?

Or is this just an argument over the use of the word "typo" instead of "mistake"? That wouldn't be very useful, IMO.

Re: Excellent succinct breakdown of the xz mess, from an OpenBSD developer

#12

Post by dvh »

AwaAwa wrote: post #10
brenoguim wrote: post #9

> It does not plausibly pass for a typo because no typical editing glitch will leave a '.' character there. I would certainly attribute that to a typo if I was reviewing the code.

> The syntax error is a single period '.' as the first character on an otherwise empty line of C code. I believe the point is that given this context, it could/should not be construed as a typo.

Very common if you copy paste (using mouse buffer) from midnight commander text editor or any other editor that highlights trailing spaces as dot.

Re: Excellent succinct breakdown of the xz mess, from an OpenBSD developer

#13

Post by lenerdenator »

AwaAwa wrote: post #10
brenoguim wrote: post #9

> It does not plausibly pass for a typo because no typical editing glitch will leave a '.' character there. I would certainly attribute that to a typo if I was reviewing the code.

> The syntax error is a single period '.' as the first character on an otherwise empty line of C code. I believe the point is that given this context, it could/should not be construed as a typo.

I take it they're not forcing contributors to lint code, then?

Re: Excellent succinct breakdown of the xz mess, from an OpenBSD developer

#14

Post by zokier »

waynesonfire wrote: post #5

> Liblzma ends up dynamically linked to sshd because of a systemd-related extension added by many Linux packagers that pulls in liblzma as an unrelated dependency kinda wish this was unpacked a bit more, why exactly is a service executable dynamically linking to a library without using any of its symbols or functions, because of systemd. and a follow up if some openbsd folks can comment. over the years i've read abou…

Libsystemd is a grabbag utility library, and completely optional:

> The libsystemd library provides functions that allow interacting with various interfaces provided by the systemd(1) service manager, as well as various other functions and constants useful for implementing services in general.

https://www.freedesktop.org/software/systemd/man/latest/libs...

E.g. the service readiness protocol is pretty trivial to implement yourself if you don't want to pull libsystemd as dep.

Re: Excellent succinct breakdown of the xz mess, from an OpenBSD developer

#15

Post by logicprog »

waynesonfire wrote: post #5

> Liblzma ends up dynamically linked to sshd because of a systemd-related extension added by many Linux packagers that pulls in liblzma as an unrelated dependency kinda wish this was unpacked a bit more, why exactly is a service executable dynamically linking to a library without using any of its symbols or functions, because of systemd. and a follow up if some openbsd folks can comment. over the years i've read abou…

> kinda wish this was unpacked a bit more, why exactly is a service executable dynamically linking to a library without using any of its symbols or functions, because of systemd.

If I recall correctly based on what I've read about this — I believe from the original mailing list post that noticed the vulnerability — it's because under certain circumstances in order to enable certain functionality you might want sshd to be able to talk to systemd, so distros often patch sshd with code to do that. But obviously you need a library to implement actually speaking system's protocol, and as it happens, the easiest way to do that is to include the entirety of libsystemd, since it has functions for doing that, even though there are at least two other libraries that implement just the communication functionality and are actually designed for non-systemd programs to use. The problem with that idea being that libsystemd, being a whole standard library for all systemd-related functionality that is probably mostly designed for use by programs in the tightly integrated systemd family and as a reference imenentation, also includes a lot of other code, including code that has to deal with compression that depends on liblzma, even though all of that is never used by sshd, because it only uses the small subsection of the library it needs.

Re: Excellent succinct breakdown of the xz mess, from an OpenBSD developer

#16

Post by throwanem »

deskamess wrote: post #6

There really is an exception to every rule. "Do not attribute to malice that which can be adequately explained by neglect, ignorance or incompetence" and then you come across: > The stage 0 shell snippet looks at first glance like a plausible part of > the poorly readable autoconf/automake tooling.

None of those adequately explain these commits. This is no exception to Hanlon's razor.

Re: Excellent succinct breakdown of the xz mess, from an OpenBSD developer

#17

Post by MarkSweep »

waynesonfire wrote: post #5

> Liblzma ends up dynamically linked to sshd because of a systemd-related extension added by many Linux packagers that pulls in liblzma as an unrelated dependency kinda wish this was unpacked a bit more, why exactly is a service executable dynamically linking to a library without using any of its symbols or functions, because of systemd. and a follow up if some openbsd folks can comment. over the years i've read abou…

The problem was that SSH linked to libsystemd. Libsystemd linked to xz. So the library got pulled in transitvly. Here are two recent commits that fix the problem by reducing dependencies:

https://salsa.debian.org/ssh-team/openssh/-/commit/cc5f37cb8...

https://github.com/systemd/systemd/pull/31550

Re: Excellent succinct breakdown of the xz mess, from an OpenBSD developer

#18

Post by amiga386 »

waynesonfire wrote: post #5

> Liblzma ends up dynamically linked to sshd because of a systemd-related extension added by many Linux packagers that pulls in liblzma as an unrelated dependency kinda wish this was unpacked a bit more, why exactly is a service executable dynamically linking to a library without using any of its symbols or functions, because of systemd. and a follow up if some openbsd folks can comment. over the years i've read abou…

sshd is started by systemd.

systemd has several ways of starting programs and waiting until they're "ready" before starting other programs that depend on them: Type=oneshot, simple, exec, forking, dbus, notify, ...

A while back, several distro maintainers found problems with using Type=exec (?) and chose Type=notify instead. When sshd is ready, it notifies systemd. How do you notify systemd? You send a datagram to systemd's unix domain socket. That's about 10 lines of C code. But to make life even simpler, systemd's developers also provided the one-line sd_notify() call, which is in libsystemd.so. This library is so other programmers can easily integrate with systemd.

So the distro maintainers patched sshd to use the sd_notify() function from libsystemd.so

What else is in libsystemd.so? That's right, systemd also does logging. All the logging functions are in there, so user programs can do logging the systemd way. You can even _read_ logs, using the functions in libsystemd.so. For example, sd_journal_open_files().

By the way... systemd supports the environment variable SYSTEMD_JOURNAL_COMPRESS which can be LZ4, XZ or ZSTD, to allow systemd log files to be compressed.

So, if you're a client program, that needs to read systemd logs, you'll call sd_journal_open_files() in libsystemd.so, which may then need liblz4, liblzma or libzstd functions.

These compression libraries could be dynamically loaded, should sd_journal_open_files() need them - which is what https://github.com/systemd/systemd/pull/31550 submitted on the 29th February this year did. But clearly that's not in common use. No, right now, most libsystemd.so libraries have headers saying "you'll need to load liblz4.so, liblzma.so and libzstd before you can load me!", so liblzma.so gets loaded for the logging functions that sshd doesn't use, so the distro maintainers of sshd can add 1 line instead of 10 to notify systemd that sshd is ready.

Re: Excellent succinct breakdown of the xz mess, from an OpenBSD developer

#19

Post by mrspuratic »

waynesonfire wrote: post #5

> Liblzma ends up dynamically linked to sshd because of a systemd-related extension added by many Linux packagers that pulls in liblzma as an unrelated dependency kinda wish this was unpacked a bit more, why exactly is a service executable dynamically linking to a library without using any of its symbols or functions, because of systemd. and a follow up if some openbsd folks can comment. over the years i've read abou…

AFAICT libsystemd is the "blessed" way of a daemon interacting with systemd and its notification protocol. libsystemd is something of a kitchen sink library now, it wasn't always so: there used to be separate libsystemd-daemon that had the functions a daemon might use (and not the ones it won't, like compression wrappers for functions in external libraries). https://lwn.net/Articles/587373/

A lot of things (e.g. OpenSSH, Apache) don't use it natively, you won't tend to see it in the official sources of non-Linux specific software. Package maintainers who choose to foist^H^H^H^H^Huse systemd add it to those packages. It's quite fun working out why Apache hangs when you port a config and don't realise it needs mod_systemd loaded to stop systemd stamping on it.

There's an extra security wrinkle too: lazy symbol resolution is not the done thing now, it's preferred that symbols are resolved at loading time so important chunks of memory can be protected from other types of attacks (more clarity in one of the preceding articles: https://research.swtch.com/xz-script ).

Re: Excellent succinct breakdown of the xz mess, from an OpenBSD developer

#20

Post by klyrs »

lenerdenator wrote: post #13
AwaAwa wrote: post #10

Earlier quoted context omitted.

> The syntax error is a single period '.' as the first character on an otherwise empty line of C code. I believe the point is that given this context, it could/should not be construed as a typo.

I take it they're not forcing contributors to lint code, then?

Does your linter look for syntax errors in the content of strings? If so, how would you design a unit test meant to capture syntax errors?
Post reply on HN