Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cull your dependencies

tomrenner.com

Post reply on HN
Print view View original

111–120 of 132 posts

Re: Cull your dependencies

#111

Post by thephyber »

cvccvroomvroom wrote: post #76
EdSchouten wrote: post #13

I don't think there's an issue with depending on libraries that are big. If the library is just a big bag of classes from which you cherry-pick the ones you need, then the extrapolation on the number of bugs as mentioned in the article under "By the numbers" is meaningless. Why care about bugs in code you don't use if it doesn't even end up getting linked into your program? In modern languages like Go or Rust, it wil…

HeartBleed, Spring4Shell, LogJam, Struts, Jetty and many more beg to differ. More LoC is always a greater attack surface, regardless of development trustworthiness. Minimize code ruthlessly.

It’s less about lines of code and more about scope of functionality.

> Minimize code ruthlessly.

Minimize functionality ruthlessly.

> More LoC is always a greater attack surface

More… than what? What does the counter factual look like?

If I only care about 1 application in a vacuum, reducing LoC is not terribly difficult. If I run my application on any modern OS, I depend on thousands of applications, daemons, libraries, and a kernel. I would far rather their developers take reasonable efforts to import common libraries when appropriate. The aggregate LoC of an ecosystem is more important than the LoC of a single application.

Also, telling people the metric of relevance is LoC is wrong and will lead people to game the metric, losing sight of the actual goal of code quality. There are infamous examples of Perl code golf; they optimize for LoC, but aren’t at all useful for code quality or security.

Far better to expand LoC a reasonable amount in favor of developer readability and to reduce complexity.

Re: Cull your dependencies

#112

Post by thephyber »

mcdonje wrote: post #100
nemothekid wrote: post #91

Earlier quoted context omitted.

The corollary to "Minimize code ruthlessly" is "Roll your own X". Sure Heartbleed was bad, but are you going to write your own TLS library? IMO, the OP post has an unfounded sense of hubris. Everyone else's code is bad except for me, who only writes minimal code with no exploits.

Libraries are generally larger because they are usually written for more use-cases. If your program is only referencing a small portion of a library, then rolling your own doesn't mean rewriting the library. Your code will be more minimal because it's only written for your program. Assuming you're fallible and write code with exploits like everyone else, your program will probably have different exploits from mine. A…

This comment inflates the upside risk (risk that every developer reinventing the wheel creates lower chance of hack of their code succeeding) and discounts the downside risk (the increased costs associated with elimination of specialization/consolidation of code into reusable libraries).

> Instead, they'll have to tailor an attack for your code base.

This misstates the actual threat.

Assume detection of exploits is automated (even if it isn’t completely automated now). Assume exploit code is modularized and passed around on hacker markets, because they are.

Your recommendation only increases the cost to exploit by a small factor, but it also increases the inevitable costs of legitimate usage by possibly a much larger factor (patching the inevitable bugs which are found). Because developers aren’t specializing (in how to efficiently+safely write logging code) bugs both in the designs phase and implementation phase will be higher than in a counterfactual specialization world.

Re: Cull your dependencies

#113

Post by eatonphil »

pornel wrote: post #52
eatonphil wrote: post #8

Earlier quoted context omitted.

> Good luck doing that in JS which has zero standard library on the browser. The article is saying something more like vendor your dependencies (and cut out the stuff you don't use within dependencies). > I prefer to not import tiny libraries but adopt the code into the codebase. Yep that's what the article is saying.

I don't get why people find vendoring valuable. If you vendored log4j, you'd get the same exact bug in your vendored version, and you'd need more work to pull in the fix. Perhaps people imagine that if they vendor they'll review all the code they pull in, but I've never seen it happen in practice beyond "LGTM". It wouldn't have found the log4j vulnerability, and could overlook even intentionally malicious code if onl…

Why are so many people using log4j or any complex logging library in the first place?

Re: Cull your dependencies

#114

Post by thephyber »

koblas wrote: post #83
wcedmisten wrote: post #4

It would be nice if there were a way to quantify the value of your dependencies. If you have a dependency with 1,000 LoC and your application is utilizing 800 of them, that seems like a good reason to use the dependency. You're (hopefully) getting unit tests, documentation, and public exposure of the code (bugfix opportunities) for "free" If you have a dependency with 1,000,000 LoC and you only need 1,000, that indic…

This makes me think about the quality of the dependancy. If you could evaluate packages for things like: * Test coverage -- that's not a metric on NPM * Code practices -- what's the review history * Issue velocity -- hard metric, lots of features vs fixes * Hygiene -- for many languages is typing enforced / validated I'm sure there are lots of other metrics, but so many times you're just evaluating two packages based…

Agree that this is the largest issue with code quality.

Knowing when and which package to import, given the incomplete data points we have now.

Stars/downloads are a popularity contest. At some point, people mostly vote for the candidate who is most likely to win (causing this to be self-reinforcing) , not the one with the best ideas.

The stability+sustainability of the development team, the signals of consistent quality (eg. Code linting, code quality audits, bug bounty program participation, public security audits, good design documents, automation of builds, testing methodology and test coverage).

Re: Cull your dependencies

#115

Post by thephyber »

kerblang wrote: post #77

I think a lot of naysayers here are missing the point: > The underlying Log4J library is 168,000 lines of code. I would find it difficult to invent a logging system so exotic, even if somebody paid me to. People are ignoring the incredible size of their real footprint. My 4000-line microservice is actually my 3,000,000-line macroservice and every single one of those lines is a potential trouble spot for security bugs…

The counter factual might suffer from Chesterton’s Fence. If you don’t know why the framework implemented “all sorts of things during boot up without our knowledge”, why do you believe that you wouldn’t need to do something similar if you were to rewrite similar code.

The core problem is the knowledge handoff of a library/ framework maintainer. An end user should be able to identify whether the library gives them enough specialization value or whether substituting it for a little elbow grease is “worth the squeeze”. In practical terms, it is always an unknown and depends on the quality of the end developer x the complexity of the task.

Re: Cull your dependencies

#116

Post by thephyber »

bayesian_horse wrote: post #64
a15971 wrote: post #56

Earlier quoted context omitted.

It's not that '... developers ... are lazy, and prefer to write as few lines of code as possible, sticking rigidly to the principle of "not reinventing the wheel"'. They don't do that because they are lazy. They do that because of competitive pressure. In SW development, in most cases, particularly in enterprise development, "the fastest person wins". Whoever moves fast and delivers fast will get to do more projects…

Reinventing the wheel is also decidedly unfulfilling.

“Fulfillment” in a developer’s programming tasks is not a valuable work product. Choosing whether to reinvent the wheel or not is ultimately a business decision that too many developers internalize.

If your product team and management team are blindsided by tech debt or liabilities that you add to the code base because you decided to quietly roll your own crypto (or any other OWASP vulnerability), you are substituting your short term enjoyment for the livelihoods of your coworkers.

Re: Cull your dependencies

#117

Post by thephyber »

All of my points have already been stated.

The author’s heuristic is too simple.

Simply rolling your own is not smart because there is a lot of detail (planning, implementation, testing, bug reporting, updating to work with different browsers/OSes/locales) that someone specialized in creating. If you are sure you don’t need to benefit from that specialization effort, it might be worth it to roll your own.

On the other side of the ledger, there is a lot of uncertainty in choosing the right library, predicting when upstream changes might cause you heartburn (eg. short notice broken API) downstream. Also, predicting what hidden features the libraries have that you don’t want or need (log4j’s formatting RCE, Java Spring-Web deserialization) or how mature the library’s development/testing/maintenance is.

Making these data points more standard and transparent (is this part of “software supply chain bill of materials” proposals?) might help better inform these decisions.

Re: Cull your dependencies

#118

Post by woojoo666 »

wcedmisten wrote: post #4

It would be nice if there were a way to quantify the value of your dependencies. If you have a dependency with 1,000 LoC and your application is utilizing 800 of them, that seems like a good reason to use the dependency. You're (hopefully) getting unit tests, documentation, and public exposure of the code (bugfix opportunities) for "free" If you have a dependency with 1,000,000 LoC and you only need 1,000, that indic…

It's not a way of detecting it but what about dead code elimination, like Javascript's tree shaking [1]?

Of course if your dependency is a spaghetti, tree shaking wouldn't do much but neither would an analysis tool. Poorly architected code with no separation of concerns, will cause every entrypoint to touch every LOC

[1]: https://developer.mozilla.org/en-US/docs/Glossary/Tree_shaki...

Re: Cull your dependencies

#119

Post by nemothekid »

jjav wrote: post #107
nemothekid wrote: post #91

Earlier quoted context omitted.

The corollary to "Minimize code ruthlessly" is "Roll your own X". Sure Heartbleed was bad, but are you going to write your own TLS library? IMO, the OP post has an unfounded sense of hubris. Everyone else's code is bad except for me, who only writes minimal code with no exploits.

> The corollary to "Minimize code ruthlessly" is "Roll your own X". Sure Heartbleed was bad, but are you going to write your own TLS library? But using "don't roll your own crypto" doesn't generalize to using every possible library to avoid writing any code. Down that path lies the madness of leftpad. Crypto and security protocol implementations are exceptionally difficult to write in a way that avoids all exploits,…

My counter-point is twofold. Everyone understand not to roll your own crypto not because developers intrinsically know crypto is hard but because of sustained marketing by cryptographers. It's an easy counter to point to when people swing too far into NIH. Following that I think a lot of people don't realize when they are in over their heads; it happens a lot in software estimation. It's easy to look at something like logging and say "I can just write printf, why does logging-library-x have so much bloat" without realizing how complex a good logger can get. Software architecture is judged by how easy it is to maintain over time; what is small amounts of code today can easily balloon into something unmanageable.

At the same time, there's a lot of hubris that goes into the statement "Every third-party library you add is an additional source of bugs, unmaged surprises", but somehow the code you write was typed by God himself.

Everyone points to leftpad madness of dependencies despite the fact that the dominos leading up to leftpad was the work of one very dedicated individual. Despite other languages having the same kind of simple dependency management never having anywhere close to the same issue (like Ruby, on which npm was based). Regardless, the OP doesn't use leftpad as an example; he uses log4j. I agree that log4j had insane defaults, and that's on the maintainers, but the implication is that you should roll your own logging facade and that's something where I wouldn't want to work on a project where all the logging is done through printf (except in probably an embedded environment) or maintain an in-house logging library because the original writer thought it would be "just small amounts of code".

Re: Cull your dependencies

#120

Post by ratww »

thephyber wrote: post #115
kerblang wrote: post #77

I think a lot of naysayers here are missing the point: > The underlying Log4J library is 168,000 lines of code. I would find it difficult to invent a logging system so exotic, even if somebody paid me to. People are ignoring the incredible size of their real footprint. My 4000-line microservice is actually my 3,000,000-line macroservice and every single one of those lines is a potential trouble spot for security bugs…

The counter factual might suffer from Chesterton’s Fence. If you don’t know why the framework implemented “all sorts of things during boot up without our knowledge”, why do you believe that you wouldn’t need to do something similar if you were to rewrite similar code. The core problem is the knowledge handoff of a library/ framework maintainer. An end user should be able to identify whether the library gives them eno…

> The counter factual might suffer from Chesterton’s Fence. If you don’t know why the framework implemented “all sorts of things during boot up without our knowledge”, why do you believe that you wouldn’t need to do something similar if you were to rewrite similar code.

I really hate those though-terminating cliches HN loves to drop from time to time. If your Chesterton's Framework (or library) is doing anything important at all during boot (or at any time, really), it should be very well documented or extremely obvious, otherwise it's a huge security risk, period. "Random Chesterton Fence shit happening" is how we got Heartbleed (did we really need the heartbeat?). It is how we got Log4Shell (did we really needed arbitrary access to servers?). It's how we still get lots of weekly Wordpress CVE.

About the "rewrite" part: the irony is that people who would be able to "rewrite similar code" are the ones who do actually know what frameworks do during initialisation. Notice that I said "be able" instead of "dare". That's because actually writing this kinda code is non-trivial and requires previous knowledge and study.

The scenario where a person is actually able to rewrite an important part of a framework that is actually production-ready without knowing what it entails beforehand is completely unrealistic.

The story of the mythical cowboy coder that managed to accidentally reimplement Rails at work is just that: a myth. What the mythical cowboy coder most certainly created was a simulacrum of a Framework. It probably has lots of useless OOP patterns, but deep down he's using Sinatra's router, and his "ORM" (if it's really an ORM) is just a wrapper around ActiveRecord. Why? Because writing a Router and an ORM are hard. Faking structure not so much.

It's frankly tiring that lots of people jump to those absurd arguments to defend the abuse of third-party dependencies as if it were mana from heaven made with the utmost care. It's not, we have to be realistic: lots of them are absolute shite.

Post reply on HN