221–230 of 335 posts
Earlier quoted context omitted.
My Fortune 500 company has to pay a third party to pay for Google Maps, because we can't pay a supplier with a credit card. If we can't find a way to pay Google, there is no hope for some random developer in Nebraska.
Just because a dev lives in the Midwest doesn’t mean they’re idiots compared to coastal elites. The hate/mockery/minimizing of anyone not from a major coastal city is really gross. Screw you.
Thanks to all Log4j developers. As users of your free software since ~20 years, we're sure that, if this vuln disclosure had been better and there were no sign of widespread in-the-wild exploitation, you would have done a better job at patching it, and a single release would have been enough. We won't hold a grudge against you; open source means collaboration, and you don't blame hard-working people that give away th…
I lay any blame squarely at the feet of IT security of large organisations that were entirely unprepared to update a widely used dependency that wasn't an operating system or a runtime. Windows, Linux, Java, and .NET are all commonly updated, but "modules" aren't. This wasn't just a predictable scenario, it was predicted. Or more accurately, it has occurred already repeatedly in the NPM ecosystem, but for some myster…
Earlier quoted context omitted.
The source code is not what is deployed . There is typically no link back from a deployed system to its source, even if you compiled the binary within your organisation. If it came from an external organisation, things are exponentially harder. Modern deployment systems are largely "one-way", with no way to trigger a full recompile from the deployment end of things. If you have a VM with "SomeRandomBinary.exe" runnin…
Go 1.18 onwards will solve this problem: https://utcc.utoronto.ca/~cks/space/blog/programming/GoVersi... > The go command now embeds version control information in binaries including the currently checked-out revision and a flag indicating whether edited or untracked files are present.... Additionally, the go command embeds information about the build including build and tool tags (set with -tags), compiler, assemble…
Earlier quoted context omitted.
My Fortune 500 company has to pay a third party to pay for Google Maps, because we can't pay a supplier with a credit card. If we can't find a way to pay Google, there is no hope for some random developer in Nebraska.
Just because a dev lives in the Midwest doesn’t mean they’re idiots compared to coastal elites. The hate/mockery/minimizing of anyone not from a major coastal city is really gross. Screw you.
Nebraska is a reference to an XKCD Comic (https://xkcd.com/2347/). The idea is that there are nameless people all over the world contributing to critical open source infrastructure.
Back to the point.
Running Google Maps on one of my low traffic sites, would generate at most, several dollars a month in charges. It would be a rounding error in our budget, and we can't pay it. Our purchasing rules are not flexible enough to work with google.
Paying Google is hard enough, an independent oss developer with no company would be impossible. I would get laughed at if I suggested throwing a few bucks at a developers Patreon.
It's amazing that this library is used by so many large enterprises yet none of them made an effort to thoroughly audit the code (actually, it looks like Alibaba reported the first one. But the point stands that the code was used for so long and by so many.)
It's amazing that this library is used by so many large enterprises yet none of them made an effort to thoroughly audit the code (actually, it looks like Alibaba reported the first one. But the point stands that the code was used for so long and by so many.)
It's just not as simple as "security audit finds all the vulnerabilities, then you fix them." You invest X in the review, you get the results that X/(hourly rate) finds. This is a lot of software with a ton of configurability-- that's a lot of variations to review and test.
Now that someone found the first lump of gold and gave it away, there are thousands of eyes searching for the next one. These recent findings are all abuses of this same chain of functionality, just along different sets of settings. In another month we might have half-a dozen more of varying severity and scope. That _still_ won't prove that the overall library is then safe, but we will probably have a a little more confidence in this particular bit of crazy template formatting flexibility. Maybe not as much as we had had three weeks ago, but more than we do now.
Earlier quoted context omitted.
The source code is not what is deployed . There is typically no link back from a deployed system to its source, even if you compiled the binary within your organisation. If it came from an external organisation, things are exponentially harder. Modern deployment systems are largely "one-way", with no way to trigger a full recompile from the deployment end of things. If you have a VM with "SomeRandomBinary.exe" runnin…
> The source code is not what is deployed. There is typically no link back from a deployed system to its source, even if you compiled the binary within your organisation. Do you not have this? Our docker images are tagged with the git hash they were built from, so at any point, for any of our envs, I can pull up the lock file of that build. Our deployment config also describes everything that is running the relevant…
Imagine you are Mr SecOps guy, and you've just ran some sort of Log4j tool across literally three thousand servers. Of those, several hundred came back positive.
Those included about a dozen flavours of Linux, a smattering of manually built(!) containers, and every version of Windows from 2008 R1 to 2022. Most of the code was built by third parties, some under support contract, some not. Most was built and installed manually, with developers using RDP or SSH to edit config files an whatnot directly on servers.[1]
So what you have now is literally just a string to a path, something like "D:\apps\foo\bar\baz\libs\stuff\thingie\log4j-core.jar" or the Linux equivalent.
Now what?
No, seriously, now what do you do? You're in SEC OPS. Not dev ops. You're certainly not in the dev team with access to the Git repo of some random vendor product like Tableau, or JIRA, or whatever[2]. You didn't deploy it. It got installed by a contractor during a short-term project three years ago.
A random hash string is totally useless to you. Even a repo URL and the commit hash will more than likely just end with an "Access Denied" URL, assuming you even have a network route to the Super Secure SCM Server.
There is no way you can figure out who needs to do what to make this go away. Not at this kind of scale at any rate. Not without first-class automation for literally everything. Which you can't have, because third-party software just doesn't play nice with any one tooling you'd like to use.
Containerisation? Bahaha... haha... snort. You're dealing with vendors that literally advertise "now with 64-bit support" and are unable to comprehend the concept of unattended command line installers. Vendors that insist on USB dongles for licensing. License that expire. Annually. And are tied to CPUID values. And on, and on.
[1] Oh, you think you can dictate release methodologies to these people? They're bureaucrats and they play politics better than you. Any word of changing their workflow in any way will immediately bring their boss, their bosses' boss, and maybe a few more levels up down upon your lowly head. You will have people literally screaming at you that your fanciful notions of build pipelines is "too much" and would "impact the work". That's the end of the conversion. I said THE END, and good day sir.
[2] Just get them to update it under their support contract? Ha-ha. Ha. Haaaa... We had vendors straight up lie about the vulnerability of their software. Then another vendor said that updating the JVM already mitigates the issues and hence they're not going to release an update. (Narrator: JVM updates aren't sufficient.)
It's amazing that this library is used by so many large enterprises yet none of them made an effort to thoroughly audit the code (actually, it looks like Alibaba reported the first one. But the point stands that the code was used for so long and by so many.)
I did this years ago with an open-source and reasonably popular non-Apache library. The results were so horrifying my boss quickly concluded such audits were outside our responsibility and the time I spent on the audit was written off. It was especially difficult for us because we’d shipped so much code that used the library, and replacing the library was unthinkable.
Earlier quoted context omitted.
Yeah, doing so much more is a bug, not a feature. > It’s dynamically configurable It's a logging library. You don't reconfigure logging, you just make another logger and use it. And by the way, you take the configurations from your global configuration system, "dynamic configurable" is a feature of the configuration system, not of the logging. > supports a wide range of output formats and destinations Yeah, like netw…
> It's a logging library. You don't reconfigure logging, you just make another logger and use it. Prior company had a shared plugin that let you turn up the logging for a given logger remotely for 30 minutes (it self reset). Extremely helpful for incident debugging. Now we can have an interesting discussion about whether that capability is worth the complexity, but that’s a very different discussion than unilaterally…
And you really believe this should be a feature of you logging library, and not of you configuration system?