Log4j: The pain just keeps going • Hacker News Board

Re: Log4j: The pain just keeps going

#221

Post by secabeen » Wed, Jul 20, 2022, 8:47 PM UTC

Kalium wrote: post #209

BrainVirus wrote: post #37

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware. If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues. The current software engineering paradigm has no meaningful answer to this , no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the parad…

> The current software engineering paradigm has no meaningful answer to this, no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the paradigm, but people in our industry seem to be only doubling down. On the contrary, I believe we do. The answer is ongoing maintenance . The basic problem is that people persist in looking at software as a fundamentally mechanistic…

So then what do you do with software that doesn't meet this standard? Software that works, that meets a need you have, but that is a fixed artifact, or at least will be in the foreseeable future. Do you refuse to use said software, and instead choose a less useful alternative that is actively developed?

Re: Log4j: The pain just keeps going

#222

Post by kelnos » Wed, Jul 20, 2022, 8:48 PM UTC

stonemetal12 wrote: post #168

gnulinux wrote: post #151

Earlier quoted context omitted.

Quite the opposite, we have very safe languages now in 2022. Any software that's not written in Rust or Haskell has to explain the trade-off of being written in an unsafe language. Of course the industry will still use unsafe languages, but we lack the awareness that we're intentionally writing garbage software in garbage languages that should be treated as radioactive material in any context where safety matters. If…

These Log4j issues aren't real hardcore exploits. They wrote a logger that could run arbitrary code, then were surprised when attackers figured out how to run arbitrary code. It isn't a language problem, it is a developer practices problem. Developers code like security isn't a thing then are confused when their programs aren't secure.

> It isn't a language problem, it is a developer practices problem. Developers code like security isn't a thing then are confused when their programs aren't secure.

I'd prefer that the "fix" for this isn't "require that millions of developers around the world all change their behavior, and never screw up in the future".

Languages can still protect against these sorts of bugs. Effect systems can describe as a part of the type system or function signature what kinds of things a function can do. For example, you could have a logging function where you know that the only side-effect is that it can write to stdout or stderr, or a file. Running arbitrary commands would not be allowed, and would cause the program to fail to compile.

As a real-world example, I've used the Slick SQL library in Scala, and the query type has a type parameter for whether the query is an read or a write. It's easy to build a simple abstraction on top of your data layer to never allow a write to go through, for example, if that's something that's useful to prevent bugs. I think one thing I did with that was set things up so callers could not accidentally write to MySQL replicas; only reads could go to them, and writes had to go to the primaries.

Granted, it takes discipline to build and use such a system, especially one that has such fine-grained effects. (I would expect most are more like a binary "does not do I/O" or "does I/O".)

Re: Log4j: The pain just keeps going

#223

Post by hnov » Wed, Jul 20, 2022, 8:57 PM UTC

BrainVirus wrote: post #37

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware. If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues. The current software engineering paradigm has no meaningful answer to this , no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the parad…

Big-tech has solved this internally. Everything down to the linux kernel they're running in prod is versioned and has people maintaining it and interacting with upstream (if any). This is obviously a full time job for at least a person per dependency on average.

Re: Log4j: The pain just keeps going

#224

Post by kelnos » Wed, Jul 20, 2022, 8:58 PM UTC

cesarb wrote: post #15

0xbadcafebee wrote: post #5

Earlier quoted context omitted.

It's funny how every Go project seems to statically compile in the same libraries. If just one of them has a serious vuln, we're talking nearly every Go project in the world having to be patched and recompiled, or upgraded with potentially breaking changes. And as we know from Log4j, that can be incredibly difficult. Just finding all the affected Go apps will be a nightmare, and patching will not be as simple as "rep…

> Just finding all the affected Go apps will be a nightmare This isn't even a new situation. We had the exact same problem some time ago with zlib: it was also embedded everywhere , and just finding all the affected apps was a nightmare. It's the reason most Linux distributions came to deeply dislike bundling libraries, instead of using a dynamically linked shared copy from a separate package. Unfortunately, it seems…

> It's the reason most Linux distributions came to deeply dislike bundling libraries

Is it, though? Seems like it would be trivial for a distribution maintainer to write a script to go through the dependency trees of every package, and then automatically rebuild any that depend on the vulnerable library. I don't think discovery is the problem.

I suspect this is really a legacy of when disk space was not so cheap; dynamic linking means smaller binaries (usually). And, even today, many distros would probably prefer their users to only download a 2MB library update from them than have to download 500MB worth of updated programs. Bandwidth is still not as cheap as we'd hope.

Edit: just realized you're probably talking about software packages vendoring in their own version of a library, not about a package statically linking with a system-provided copy of the library.

Re: Log4j: The pain just keeps going

#225

Post by layer8 » Wed, Jul 20, 2022, 9:03 PM UTC

twic wrote: post #216

layer8 wrote: post #165

Earlier quoted context omitted.

> Then apache decides to put new people on log4j, do a backward incompatible v2 design that nevertheless is worse than slf4j. Why? Because the original Log4j author wanted to do his own thing for a Log4j successor (slf4j/logback), and the Apache Log4j maintainers also had ideas about a next-gen framework and wanted to do that. That’s Open Source, everyone can do their own thing if they like, and some things will get…

I use java.util.logging in a few apps, to cut down on dependencies, and i consider it to be fairly braindead. The fundamentals are okay, to the extent that they are copies of log4j, but there are a bunch of little things that just aren't done right, like: - logging methods taking arrays rather than varargs (this would be a trivial change!) - there being no way to create a SimpleFormatter with a custom format in code…

Those are cosmetic issues, in my book. The basic architecture is sound.

In any serious application you will want to have a central logging wrapper anyway (I always use one), which automatically takes care of the varargs issue.

The implementation of SimpleFormatter is 30 lines that you can easily copy and adjust as needed, including with your own custom format.

Regarding logger names, I always define an explicit set of logger names for the application, because that makes the logger names independent from implementation-detail subpackages, and any operations-side log filtering based on logger names will remains stable, regardless of how you refactor your implementation into Java packages. Basically, I see the logger names as part of the external interface of the application, and therefore I want to have them decoupled from the internal package structure, so that mere refactorings don't break that interface. You can still easily find all program locations that log to a specific logger, for example by Find Usages on the respective static Logger constant.

Re: Log4j: The pain just keeps going

#226

Post by kelnos » Wed, Jul 20, 2022, 9:04 PM UTC

WesolyKubeczek wrote: post #130

Beltalowda wrote: post #60

Earlier quoted context omitted.

That will only work on Debian. And that "single file" requires also the libfoo-1.5.deb file (must be >=1.5, not 1.4 and not higher than 2.0!) etc.

The thing about "single file deployment" is that it shouldn't have external requirements, otherwise it's not a single file deployment anymore.

... which is why the parent is (IMO correctly) claiming that the grandparent's assertion is incorrect that a Debian package is a "single file deployment".

Re: Log4j: The pain just keeps going

#227

Post by efsavage » Wed, Jul 20, 2022, 9:04 PM UTC

secabeen wrote: post #221

Kalium wrote: post #209

Earlier quoted context omitted.

> The current software engineering paradigm has no meaningful answer to this, no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the paradigm, but people in our industry seem to be only doubling down. On the contrary, I believe we do. The answer is ongoing maintenance . The basic problem is that people persist in looking at software as a fundamentally mechanistic…

So then what do you do with software that doesn't meet this standard? Software that works, that meets a need you have, but that is a fixed artifact, or at least will be in the foreseeable future. Do you refuse to use said software, and instead choose a less useful alternative that is actively developed?

I believe the point is that no software is a fixed artifact. Regardless of the state of its development, it's all reactive to factors outside of its control, whether that's other software, users, hardware, etc.

Re: Log4j: The pain just keeps going

#228

Post by hyperman1 » Wed, Jul 20, 2022, 9:05 PM UTC

layer8 wrote: post #165

hyperman1 wrote: post #27

I never understood where log4j 2 was supposed to fit in the ecosystem. * We had log4j 1, which was good enough for most use cases and everywhere. * We had slf4j/logback, which was from the same authors, and broke backward compatibility but gave us a fundamentally better design. * We had java.util.logging, which while braindead was builtin to Java and hence available everywhere. *We had commons logging, which was neve…

> Then apache decides to put new people on log4j, do a backward incompatible v2 design that nevertheless is worse than slf4j. Why? Because the original Log4j author wanted to do his own thing for a Log4j successor (slf4j/logback), and the Apache Log4j maintainers also had ideas about a next-gen framework and wanted to do that. That’s Open Source, everyone can do their own thing if they like, and some things will get…

java.util.logging was braindead because it wanted to know the class and method doing the logging. If not provided, it threw and catched an exception and filtered the stack trace to detect its own caller. Throwing an exception was a very slow operation.

I see they did some optimizations to do this work lazily if needed. But when log4j 1 was on its peak, that wasn't done, and using the built in logger was slow enough to have measurable impact .

See https://github.com/openjdk/jdk/blob/6765f902505fbdd02f25b599... at the bottom.

Re: Log4j: The pain just keeps going

#229

Post by Veserv » Wed, Jul 20, 2022, 9:05 PM UTC

secabeen wrote: post #221

Kalium wrote: post #209

Earlier quoted context omitted.

> The current software engineering paradigm has no meaningful answer to this, no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the paradigm, but people in our industry seem to be only doubling down. On the contrary, I believe we do. The answer is ongoing maintenance . The basic problem is that people persist in looking at software as a fundamentally mechanistic…

So then what do you do with software that doesn't meet this standard? Software that works, that meets a need you have, but that is a fixed artifact, or at least will be in the foreseeable future. Do you refuse to use said software, and instead choose a less useful alternative that is actively developed?

That is solved with standard engineering practices.

If you need to make guarantees about your product that are dependent on another piece of software, then you just have your vendor produce a spec and guarantee conformance to the spec, then you audit that the spec fulfills your fixed needs and audit that the implementation conforms to the spec.

Then, if you made a mistake, your liability is limited to your mistakes instead of your mistakes and the mistakes of your dependencies.

If, on the other hand, you do not get any guarantees from your dependencies, and you fail to achieve your guarantees then that is entirely on you as you are the one transforming the absence of guarantees into guarantees.

Re: Log4j: The pain just keeps going

#230

Post by m_mueller » Wed, Jul 20, 2022, 9:13 PM UTC

secabeen wrote: post #221

Kalium wrote: post #209

Earlier quoted context omitted.

> The current software engineering paradigm has no meaningful answer to this, no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the paradigm, but people in our industry seem to be only doubling down. On the contrary, I believe we do. The answer is ongoing maintenance . The basic problem is that people persist in looking at software as a fundamentally mechanistic…

So then what do you do with software that doesn't meet this standard? Software that works, that meets a need you have, but that is a fixed artifact, or at least will be in the foreseeable future. Do you refuse to use said software, and instead choose a less useful alternative that is actively developed?

In fact yes. I‘d choose a maintained lesser library or an in-house implementation over an unmaintained one. This is also the main benefit of python, with its huge standard lib that effectively gives some maintenance guarantees.

To give you an example: XLSX parsing. For quite a while, the most performant library ‚xlrd‘ had a big scary unmaintained notice on GitHub, yet people kept recommending and using it, often unnoticed through pandas, because they don’t read docs. I‘d refuse to touch that and instead use openpyxl, and now newer xlrd releases have even completely removed xlsx support.