Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Log4j: The pain just keeps going

thenewstack.io

Post reply on HN
Print view View original

131–140 of 291 posts

Re: Log4j: The pain just keeps going

#131

Post by _dain_ »

BrainVirus wrote: post #37

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware. If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues. The current software engineering paradigm has no meaningful answer to this , no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the parad…

>The current software engineering paradigm has no meaningful answer to this, no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the paradigm, but people in our industry seem to be only doubling down.

Capability-based security is one possible answer. The Austral language is trying to make this a first-class language feature:

>The problem is that code is overwhelmingly permissionless. Or, rather: all code has uniform root permissions. The size of today’s software ecosystems has introduced a new category of security vulnerability: the supply chain attack. An attacker adds malware to an innocent library used transitively by millions. It is downloaded and run, with the user’s permissions, on the computers of hundreds of thousands of programmers, and afterwards, on application servers.

>The solution is capability-based security. Code should be permissioned. To access the console, or the filesystem, or the network, libraries should require the capability to do so. Then it is evident, from function signatures, what each library is able to do, and what level of auditing is required.

https://austral.github.io/spec/rationale-capabilities

Re: Log4j: The pain just keeps going

#132

Post by philosopher1234 »

shepherdjerred wrote: post #128
BrainVirus wrote: post #37

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware. If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues. The current software engineering paradigm has no meaningful answer to this , no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the parad…

> So the exact software you compile will depend on whether or not you can connect to a 3d party service? Do you understand the actual implications of this? This is already true with the vast majority of software using any an online software repository, e.g. Go, NodeJS, Java, Rust, etc.

Not Go

Re: Log4j: The pain just keeps going

#134

Post by hinkley »

BrainVirus wrote: post #37

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware. If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues. The current software engineering paradigm has no meaningful answer to this , no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the parad…

When I worked on a code signing app, which is arguably some of the highest stakes of almost anything I've worked on, we came around to an agreement that one ticket a month would be assigned to upgrade libraries, and we rotated that responsibility. We didn't stipulate what library, we didn't even stipulate which application in the suite (though it was assumed that you were likely to chose your primary application as the target), so long as something got upgraded.

This policy evolved, if memory serves, after a security issue was discovered with the XML library we were using. The fix was not back ported to our version, and the versions in between had multiple breaking changes, so it was a slog to fix it.

It wasn't even that we were having a production issue, because we were still in development and internal testing. Not a single 'real' signature had been generated yet. It was all test assets and test CA certs. But we had enough wisdom to put 2 and 2 together and get 4, so we tried to treat the situation as a dress rehearsal for some later bug that happened after we started playing for keeps. Almost everyone could see this was an untenable situation.

There were consequences of course, but we made a bit of lemonade in the process. Our integration tests were expanded to include a larger percentage of pinning tests for our dependencies. Given the gravity of the situation, we needed those anyway. We just now had a poster child for doing the work.

Re: Log4j: The pain just keeps going

#136

Post by xmprt »

autokad wrote: post #116
BrainVirus wrote: post #37

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware. If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues. The current software engineering paradigm has no meaningful answer to this , no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the parad…

Lets not forget that promotions are for those who build new stuff and those who maintain old stuff don't get promoted. To add further insult to injury, colleges send SWE/SDEs to the work force to learn to 'actually code', while companies have no time to train jr developers so they are just thrown in the fire and hope for the best.

At most companies I've worked for, mentoring and training junior engineers is a requirement for promotion. Also, you might not get promoted for maintenance but

1. keeping it running with a healthy userbase and demonstrating high impact does, and

2. if you know how to spin it, then maintenance work can look like "building new stuff".

Re: Log4j: The pain just keeps going

#137

Post by craigching »

jsiaajdsdaa wrote: post #55
BrainVirus wrote: post #37

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware. If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues. The current software engineering paradigm has no meaningful answer to this , no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the parad…

Better idea: stop logging.

Ironically, logging is one of the ways to help mitigate/detect security vulnerabilities.

Re: Log4j: The pain just keeps going

#138

Post by mrcarruthers »

philosopher1234 wrote: post #132
shepherdjerred wrote: post #128

Earlier quoted context omitted.

> So the exact software you compile will depend on whether or not you can connect to a 3d party service? Do you understand the actual implications of this? This is already true with the vast majority of software using any an online software repository, e.g. Go, NodeJS, Java, Rust, etc.

Not Go

Still Go. I mean there's no central repository in the way there's one for npm, and yes you can point it to any git repo as the source for your dependencies, but the reality is most are on GitHub. So your central repository is GitHub.

Re: Log4j: The pain just keeps going

#139

Post by xmprt »

alfalfasprout wrote: post #122
BrainVirus wrote: post #37

If you automatically update your dependencies all the time, you will constantly get new bugs, issues and sometimes even malware. If you don't update your dependencies all the time, you will be vulnerable to old bugs and issues. The current software engineering paradigm has no meaningful answer to this , no matter what "security experts" tell you. In a sane industry this realization would lead to a change of the parad…

We actually are running into this exact problem with running production ML workloads and despite anyone's claims it's not a solved problem. In fact, this log4j mess is a cake walk in comparison. The issue: ML workloads are very sensitive to the exact version of a framework/library used and these frameworks/libraries are not stable despite what their semantic version says. So what do you do when you have a python vuln…

Is this an issue of code stability or do these libraries have leaky abstractions that cause issues with models trained on previous version, because the library implementation changed and since ML has low explainability, it's hard to tell how the change impacts the model performance? eg. making a library run faster or use a slightly different parameter internally causes some floating point imprecision that cascades into wildly different results?
Post reply on HN