Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Log4j: The pain just keeps going

thenewstack.io

Post reply on HN
Print view View original

171–180 of 291 posts

Re: Log4j: The pain just keeps going

#171

Post by paulmd »

jsiaajdsdaa wrote: post #159
ClumsyPilot wrote: post #152

Earlier quoted context omitted.

option 1 - we and farmers, as professionals are responsible for supplying stuff that is not dangerous and works as it should Option 2 - which you are advocating - it's every man for himself and we decent into socity of warring tribes of hunter hatherer or subsitence farming at best

option 2 is the only possible state of reality, because it is never possible to guarantee that everything work in every instance let the buyer be aware

I mean, how do you know your webserver doesn't have a bug? How do you know your OS doesn't? Just write those yourself? How do you know yours doesn't? One must still verify the key functionality regardless.

But, you're not wrong, some people reach for every library and framework they can, and others focus on building the most minimal and understandable thing that will work. I'm not saying writing your own server in assembly, but, avoid unnecessary junk and magic and libraries.

It's the classic dilemma: in this world of ours, one can either return to monke or progress to crab.

Re: Log4j: The pain just keeps going

#172

Post by skrtskrt »

aclatuts wrote: post #160
alfalfasprout wrote: post #122

Earlier quoted context omitted.

We actually are running into this exact problem with running production ML workloads and despite anyone's claims it's not a solved problem. In fact, this log4j mess is a cake walk in comparison. The issue: ML workloads are very sensitive to the exact version of a framework/library used and these frameworks/libraries are not stable despite what their semantic version says. So what do you do when you have a python vuln…

I find Python is very prone to a lot of breaking changes even with minor version changes. And not just that, even minor versions have to match up specifically with another libraries minor version. It seems to be a community or ecosystem problem with Python, which is why I avoid Python as much as possible. In every other ecosystem when minor version are updated there are very rarely issues, and there are little issues…

eh, people love to break stuff on minor version changes of Go libraries to

Re: Log4j: The pain just keeps going

#173

Post by btilly »

aclatuts wrote: post #160
alfalfasprout wrote: post #122

Earlier quoted context omitted.

We actually are running into this exact problem with running production ML workloads and despite anyone's claims it's not a solved problem. In fact, this log4j mess is a cake walk in comparison. The issue: ML workloads are very sensitive to the exact version of a framework/library used and these frameworks/libraries are not stable despite what their semantic version says. So what do you do when you have a python vuln…

I find Python is very prone to a lot of breaking changes even with minor version changes. And not just that, even minor versions have to match up specifically with another libraries minor version. It seems to be a community or ecosystem problem with Python, which is why I avoid Python as much as possible. In every other ecosystem when minor version are updated there are very rarely issues, and there are little issues…

My (somewhat dated) experience says that Ruby is all the way worse than Python on this.

In part due to a culture that encouraged monkeypatching. With the result that what behavior you get depends on who last monkeypatched it. So a dependency 3 deep that loads the module you expected to load last suddenly causes the monkeypatch you were depending on to disappear...

Re: Log4j: The pain just keeps going

#174

Post by 0xbadcafebee »

chrisseaton wrote: post #161
0xbadcafebee wrote: post #121

Earlier quoted context omitted.

Imagine you are a random person in charge of a computer. You hear there's this big vuln affecting Go programs. 1. How do I find out which programs on my system are Go programs? I have a lot of programs scattered all over. Someone will have to come up with a method (probably search through $PATH for files, run `strings` on it, grep for "Go" , pray that's enough) to identify them. 2. They don't list a website, so I hav…

> How do I find out which programs on my system are Go programs? Well how are these programs getting onto your system? Presumably via some package manager, or an image builder? Don't those manage the update for you? If you're running code that you don't know who wrote, with no source code, that you can't patch, that nobody is taking responsibility for, then yeah you've got issues. But you had issues already if that's…

Most people download and install Go programs by hand since they're statically compiled, it's their main selling point

> If you're running code that you don't know who wrote, with no source code, that you can't patch, that nobody is taking responsibility for, then yeah you've got issues

25 million computers around the world still use Windows XP. It was launched 21 years ago, EOL 8 years ago. Very common actually, especially since most software in the world is not open source. People just keep using software that works.

Re: Log4j: The pain just keeps going

#175

Post by TheCapn »

fisf wrote: post #92
pclmulqdq wrote: post #65

Earlier quoted context omitted.

It takes a lot longer to build houses that are up to building codes, and bridges that have 10x safety factors rather than 2x. There's a reason we have building codes and large safety factors. Software engineers are discovering this today.

Software engineers are mostly aware of this. There is just little market demand for this level of robustness.

Engineering Code of Ethics dictate that safety of public and environment is paramount. I know that's true for my P.Eng governing body; it is written into our bylaws. I'm sure its the same for most others as well.

I think until Software Engineering catches up in this regard with traditional Engineering disciplines we'll continue to see issues arise from the software realm. As much as the self-titled "Engineers" proclaim it, there's more to engineering than design and production.

Re: Log4j: The pain just keeps going

#176

Post by paulmd »

agumonkey wrote: post #143
BrainVirus wrote: post #37

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware. If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues. The current software engineering paradigm has no meaningful answer to this , no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the parad…

And there's the tangling of dependencies. One fix in one lib may cause a bug in another that isn't maintained.

Or you can end up with conflicting versions of transitive dependencies.

Or even worse you may have a legacy "fat jar" that you can't even manage the transitive dependencies on - they're compiled in, and that's that. Something else has a version conflict? Tough shit, one or the other is going to break, and depending on how bad your build is, you may not even get an explicit choice, it may be thrust upon you by the classloader.

Sometimes you can mitigate it somewhat by encapsulating them in their own service and just firewalling the hell out of it, and modules should help the dependency conflicts somewhat, but the problem is these are basically "UXO in the backyard garden", they are a passive business risk and sooner or later you may get lucky and they go boom even though you did nothing wrong.

Despite your best intentions, sometimes you gotta update dependencies, and if you allow it to fester, then you risk that the time you have to solve the problem is during an urgent security crisis. Generally it is better to update them on your own timetable than to have it forced upon you like that. And the problems and limitations and workarounds only become worse and worse over time if you choose to actively ignore it.

Re: Log4j: The pain just keeps going

#177

Post by leeoniya »

ClumsyPilot wrote: post #112
leeoniya wrote: post #109

Earlier quoted context omitted.

> because our industry is obsessed with abstractions "we do not break userland, period" -- Linus Torvalds not breaking existing stuff is why many abstractions exist, and even why many are put in place early on, to allow for some wiggle room without having to break all the things. people tend to prefer software that continues to work, above all else.

but then someone builds a bank on top of this abstraction, and sorts of shit hits the fan.

except, it's turtles all the way down. is POSIX not an abstraction? TCP? C/gcc/llvm? glsl?

Re: Log4j: The pain just keeps going

#178

Post by apeace »

BrainVirus wrote: post #37

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware. If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues. The current software engineering paradigm has no meaningful answer to this , no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the parad…

I have a very simple solution, in three parts:

1) We use very few dependencies. We err on the side of writing our own function, library, or UI component rather than installing a new dependency, even if a "better" open-source version might theoretically be available.

2) We use a monorepo. We have two Go backends sharing the same go.mod, and three Typescript/React frontends sharing the same package.json, all in the same repo. An upgrade for one is an upgrade for all.

3) Every first Friday, I update all dependencies to the latest version. Including major versions. Yes this is a pain sometimes (like when I realized we were on Webpack 4 and the latest was Webpack 5), but if you do it consistently, it's usually not that much trouble. 1-2 hours per month on average.

In terms of security, I think this is a good middle ground. The approach has other benefits than security, too.

Re: Log4j: The pain just keeps going

#179

Post by paulmd »

gnulinux wrote: post #151
ClumsyPilot wrote: post #115

Earlier quoted context omitted.

'software written in garbage languages' well, back to the mainframe it is then

Quite the opposite, we have very safe languages now in 2022. Any software that's not written in Rust or Haskell has to explain the trade-off of being written in an unsafe language. Of course the industry will still use unsafe languages, but we lack the awareness that we're intentionally writing garbage software in garbage languages that should be treated as radioactive material in any context where safety matters. If…

In what fashion does Rust prevent you from calling valid library code that has unexpected behavior?

Re: Log4j: The pain just keeps going

#180

Post by agumonkey »

paulmd wrote: post #176
agumonkey wrote: post #143

Earlier quoted context omitted.

And there's the tangling of dependencies. One fix in one lib may cause a bug in another that isn't maintained.

Or you can end up with conflicting versions of transitive dependencies. Or even worse you may have a legacy "fat jar" that you can't even manage the transitive dependencies on - they're compiled in, and that's that. Something else has a version conflict? Tough shit, one or the other is going to break, and depending on how bad your build is, you may not even get an explicit choice, it may be thrust upon you by the cla…

it's an interesting issue technically.. if only we didn't have to suffer its reality :)
Post reply on HN