Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Log4j: The pain just keeps going

thenewstack.io

Post reply on HN
Print view View original

31–40 of 291 posts

Re: Log4j: The pain just keeps going

#31

Post by zdragnar »

0xbadcafebee wrote: post #5
WesolyKubeczek wrote: post #3

Sometimes people ask me why I'm so skeptical about software delivery techniques involving bundling and static compilation. This is a thing that people sometimes ask.

It's funny how every Go project seems to statically compile in the same libraries. If just one of them has a serious vuln, we're talking nearly every Go project in the world having to be patched and recompiled, or upgraded with potentially breaking changes. And as we know from Log4j, that can be incredibly difficult. Just finding all the affected Go apps will be a nightmare, and patching will not be as simple as "rep…

Easy, just don't write bugs. /s

That said, I think this missing the full picture- we have the same problem further down the stack, at the OS level, where old machines are running without receiving security updates on a regular basis.

I don't know that there's really much of a good answer to the problem short of "don't leave your children unattended". Unlike real kids and the real world, there's plenty of bad actors looking for unattended services to stab.

Re: Log4j: The pain just keeps going

#32

Post by LaGrange »

kevincox wrote: post #13
0xbadcafebee wrote: post #5

Earlier quoted context omitted.

It's funny how every Go project seems to statically compile in the same libraries. If just one of them has a serious vuln, we're talking nearly every Go project in the world having to be patched and recompiled, or upgraded with potentially breaking changes. And as we know from Log4j, that can be incredibly difficult. Just finding all the affected Go apps will be a nightmare, and patching will not be as simple as "rep…

I also like now "single file deployment" is considered a feature. It makes it easier for people without proper packaging and distribution tools. Of course these people without packing and distribution tools will now have a hard time finding out everywhere they have copied these binaries to.

packageblabla-1.2.3.deb is a single file.

Re: Log4j: The pain just keeps going

#33

Post by tekchip »

hedora wrote: post #17
jacobyoder wrote: post #12

Had to deal with some automated scan tool telling a client "you have log4j - you're vulnerable!" when... we didn't. A project was including a dependency on SLF4J and that project pulled in slf4j-log4j (IIRC). But it was not configured, wasn't compiled in to the final jar, and ... even it was, it was very old log4j (1.1 or 1.2 IIRC?). The vulnerability didn't affect that older version. We had to spend a week back and…

Once you realize this sort of shallow, automated audit exists to grease the wheels of some security theater operation, this sort of back and forth makes sense. False positives don't waste the security team's budget, and are the only surefire way to justify ongoing expenditures on the scanning tool.

So somehow false positives negate all the actual positives caught and corrected? The only true solution is what? Manual audit of everything by some perfect human security practitioner? I suppose the same applies to automated development tools then.

I will concede there probably are some firms out there acting poorly that way. When aren't there? Humans sigh. But by in large automation and the problems inherent are required.

Re: Log4j: The pain just keeps going

#34

Post by commandlinefan »

upupandup wrote: post #8

It's probable that there are more log4j type of exploits thats just waiting to be discovered. It's also becoming hard to tell which are private enterprises vs state sponsored cyber attack. It just seems like there's no solution. If Java is this bad imagine how much more attack surface a npm heavy project. Not only that the seeming hardware knowledge of state sponsored attackers seem sophisticated enough to gain acces…

> seems like there's no solution

There is - don't depend on anybody else's software. It just means that software development will take longer and cost more.

Re: Log4j: The pain just keeps going

#35

Post by LaGrange »

AndrewDucker wrote: post #4
neals wrote: post #2

As a small time hobby / small business server administrator (enthusiast) , what's the best practice here?

Make sure you have the time to patch/update your applications when security issues are raised.

Whoa whoa whoa, slow down, that would mean somebody in the organization would have to spend _valuable senior developer time_ on reading things.

Yes, that's jaded, but that's what I'd expect. So far I've yet to feel like a job _encouraged me_ to pay attention to security updates, accessibility standards, or anything like that. And it's quite hard to keep myself motivated to do that, when it's pretty clear most software products are disposable anyway.

Re: Log4j: The pain just keeps going

#36

Post by dspillett »

jacobyoder wrote: post #12

Had to deal with some automated scan tool telling a client "you have log4j - you're vulnerable!" when... we didn't. A project was including a dependency on SLF4J and that project pulled in slf4j-log4j (IIRC). But it was not configured, wasn't compiled in to the final jar, and ... even it was, it was very old log4j (1.1 or 1.2 IIRC?). The vulnerability didn't affect that older version. We had to spend a week back and…

> We had to spend a week back and forth with

We've had similar out of a pen test that noted we were using nginx for some of our internal dashboards/tools and the version running (1.18 IIRC) was officially EOLed upstream so a high security issue.

They initially wouldn't listen to the argument that we use stable/LTS Debian/Ubuntu release only, and those hold functionality stable (which is one of the key reasons to use them) and backport security updates where relevant. We pointed them at the package changelogs and they were convinced that there was still a vulnerability not patched but for some reason didn't want to tell us which, when we finally got that out of them it turned out to be one that was introduced in a later version so was never relevant in the first place for the one the stable repositories included.

You would think a penetration testing company would have a clue about something so common…

Re: Log4j: The pain just keeps going

#37

Post by BrainVirus »

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware.

If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues.

The current software engineering paradigm has no meaningful answer to this, no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the paradigm, but people in our industry seem to be only doubling down.

"Use a build tool to do dependency checks!"

And then what?

"Only update libraries if they have vulnerabilities."

So, should this be a manual process where I have to dig through obscure warnings every time I build something?

"No, you can automate it!"

Congrats, you've just outsourced the decision making process about which versions of libraries to use to some 3d party. You've also made your build process reliant on yet another online service.

"But it's not a hard dependency, you can still build if the 3d party service is off."

So the exact software you compile will depend on whether or not you can connect to a 3d party service? Do you understand the actual implications of this?

Etc. People with clever advice don't seem to think it through. We're fighting fragile complexity of too many tools ducktaped together by ducktaping more tools to the whole setup. Again and again and again.

Re: Log4j: The pain just keeps going

#38

Post by Natsu »

thayne wrote: post #21
jacobyoder wrote: post #12

Had to deal with some automated scan tool telling a client "you have log4j - you're vulnerable!" when... we didn't. A project was including a dependency on SLF4J and that project pulled in slf4j-log4j (IIRC). But it was not configured, wasn't compiled in to the final jar, and ... even it was, it was very old log4j (1.1 or 1.2 IIRC?). The vulnerability didn't affect that older version. We had to spend a week back and…

I thought Log4j 1.1 was vulnerable, just to a lesser extent, and there wasn't a lot of information on it because 1.x is end of life

EOL libraries are themselves vulnerabilities. It's not really good to even have this reachable on the classpath, in cases like the grandparent, you should look at removing the class from inside the JAR (or removing the JAR itself) if possible to ensure that the vulnerable code cannot get called under any circumstances.

Re: Log4j: The pain just keeps going

#39

Post by bxparks »

hedora wrote: post #24
bgro wrote: post #16

When are they going to release Log5j? Or even Log4k to address these issues?

The log4j author already did this about a decade ago. No one could understand log4j back then, so the replacement is much, much simpler, and API compatible for 99+% of use cases. Few projects seem to have bothered moving to it, probably becuase log4j is such a collosal pain to set up or modify.

What is that replacement? java.util.logging?

Re: Log4j: The pain just keeps going

#40

Post by shagie »

hedora wrote: post #22
shagie wrote: post #14

Earlier quoted context omitted.

I'm not as familiar with Go... I do know that we use a dependency scan that looks at the docker images and recognizes vulnerable versions of various Java libraries. With Go, can you look at a compiled binary in an image on the artifact repo and identify what libraries it used and if those libraries have vulnerabilities? (and it appears that my question was answered in a sibling comment)

Java's "statically link then scan" approach doesn't really work though. (Some sibling threads explain why.) Even Linux distributions where rebuilding the world is easy (like Debian) insist on breaking dependencies out into shared libaries, since tracking ad hoc vendored dependencies is just too difficult. For example, sometimes people play dynamic loading and renaming tricks to allow multiple versions of the same Jav…

I'm not sure what you mean by "statically link then scan" in the Java context. The fingerprint of the vulnerable class files are the same no matter where they are stored. They could be bundled together, or shoved in a fat jar, or exploded into all their classes -- but the class file always is distinct and identifiable.
Post reply on HN