Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cull your dependencies

tomrenner.com

Post reply on HN
Print view View original

101–110 of 132 posts

Re: Cull your dependencies

#101

Post by pitched »

wcedmisten wrote: post #4

It would be nice if there were a way to quantify the value of your dependencies. If you have a dependency with 1,000 LoC and your application is utilizing 800 of them, that seems like a good reason to use the dependency. You're (hopefully) getting unit tests, documentation, and public exposure of the code (bugfix opportunities) for "free" If you have a dependency with 1,000,000 LoC and you only need 1,000, that indic…

How much time would it take to replace those thousand lines? A week (assuming well tested and production-ready is a requirement)? If keeping it costs a couple hours of debugging once a month, the return on that week would take years to turn positive.

Re: Cull your dependencies

#102

Post by swatcoder »

smarx007 wrote: post #41

It's a good advice but it has a cost. Where is the discussion about cost? The product with less dependencies will live longer and give you better flexibility but it will cost more to build and more to maintain (incl. onboarding new engineers who need to learn their way around your custom stdlib+). It's a balanced choice but the stakeholders are not prepared to invest more. Furthermore, if the project gets cancelled,…

> will cost more ... to maintain

This is a reasonable and popular intuition, but for an enormous class of projects it turns out to be torturously false and a poor reason to choose dependencies.

How many times have you sat down on Monday to fix a bug in your product only to discover DoohickyLib 3.3.2 is no longer building correctly with the toolchain update that you just pulled in. So now you go to DoohickyLib's github page to see if it's been addressed yet.

You find that someone else reported the issue last week but the maintainers use a different toolchain themselves, and don't think this is a priority, and so they pushed back on the reporter to submit a PR if it's important to them.

Unfortunately, the reporter isn't experienced at contributing to open source and doesn't want to contribute. After a bunch of other people post "me too! when is this getting fixed?", some generous soul finally contributes a PR that should do the job.

But the maintainers are on vacation or just sick of this issue and don't respond. Finally, they reappear but aren't satisfied with the PR, so they push back on the contributor. In the meantime, that contributor just transitioned to their own fork and aren't tracking the issue anymore. So the issue has been tracked for a week and has 30 posts, and somebody shared a functional fix, but it still isn't merged and DoohickyLib still doesn't work with the toolchain you use.

It's now 1pm on Monday and you've spend most of the day trying to track down the issue and understand its status. You think about whether you can table this work until later in the week hoping that the fix gets merged into the mainline of DoohickyLib, or whether you should switch to a fork. But there's a lot of overhead to that, especially if you're on a team and need to run those kinds of ideas past a PM.

Blah, blah, blah, etc, etc, etc

This is what "maitenence" tasks look like when bring dependencies into your project. They're not really related to your project, they're not really something you have good control over, they don't feel like engineering, they often come up out of nowhere, and they're often showstoppers.

The truth is that it's very hard to anticipate where your maintenance burden will come, but when you choose to use a lot of depdendencies, you're not necessarily reducing that burden but you are making a profound choice about what it looks like.

Re: Cull your dependencies

#103

Post by nemothekid »

mcdonje wrote: post #100
nemothekid wrote: post #91

Earlier quoted context omitted.

The corollary to "Minimize code ruthlessly" is "Roll your own X". Sure Heartbleed was bad, but are you going to write your own TLS library? IMO, the OP post has an unfounded sense of hubris. Everyone else's code is bad except for me, who only writes minimal code with no exploits.

Libraries are generally larger because they are usually written for more use-cases. If your program is only referencing a small portion of a library, then rolling your own doesn't mean rewriting the library. Your code will be more minimal because it's only written for your program. Assuming you're fallible and write code with exploits like everyone else, your program will probably have different exploits from mine. A…

This is just security through obscurity. It's no different than skipping password-hashing because you think you are too small be attacked (after all you can minimize one more dependency). It works up until the point your business is actually worth anything.

Libraries are not only larger because they are written for more use-cases, but also because they cover more tested edge cases.

Re: Cull your dependencies

#104

Post by nemothekid »

spfzero wrote: post #96
nemothekid wrote: post #91

Earlier quoted context omitted.

The corollary to "Minimize code ruthlessly" is "Roll your own X". Sure Heartbleed was bad, but are you going to write your own TLS library? IMO, the OP post has an unfounded sense of hubris. Everyone else's code is bad except for me, who only writes minimal code with no exploits.

He's not comparing quality, as much as quantity. Building a small utility function, general enough for your intended use, brings in less code than adding a library dependency. That's the argument. TLS is an example of when you probably would not make that trade-off, but there are lots of other examples where it would make sense.

Sure, but the parent post is about logging - which isn't a small utility function. `printf` works for scripts up until it doesn't. When you start needing structured logging, log levels and you are debugging performance issues because of logging you start to understand why that complexity is put in place. Leftpad? Sure that's extraneous. But logging?

Re: Cull your dependencies

#105

Post by klardotsh »

netr0ute wrote: post #49
twic wrote: post #48

Earlier quoted context omitted.

Rather than reviewing any licenses, you should incorporate a license check in your build. I did this in a Gradle project recently, and it was pretty easy: https://hg.sr.ht/~twic/lambda-property-matcher/rev/53ef7eb30...

Why make it so difficult? Just make your project AGPL and stop worrying about other OSS license compatibility.

Because AGPL implies many things not all developers want associated with their project?

Yes, due to their virality, GPL-flavored licenses tend to be the "top type" that you can just cast the top-level project to and call it a day, but try convincing your corporate boss to AGPL your money-making product. That's gonna go about as well as is predictable.

Re: Cull your dependencies

#106

Post by Thiez »

netr0ute wrote: post #49
twic wrote: post #48

Earlier quoted context omitted.

Rather than reviewing any licenses, you should incorporate a license check in your build. I did this in a Gradle project recently, and it was pretty easy: https://hg.sr.ht/~twic/lambda-property-matcher/rev/53ef7eb30...

Why make it so difficult? Just make your project AGPL and stop worrying about other OSS license compatibility.

AGPL isn't even compatible with GPLv2, so you still have to worry about OSS license compatibility. So given that you have to worry anyway, you might as well pick a less obnoxious license.

Re: Cull your dependencies

#107

Post by jjav »

nemothekid wrote: post #91
cvccvroomvroom wrote: post #76

Earlier quoted context omitted.

HeartBleed, Spring4Shell, LogJam, Struts, Jetty and many more beg to differ. More LoC is always a greater attack surface, regardless of development trustworthiness. Minimize code ruthlessly.

The corollary to "Minimize code ruthlessly" is "Roll your own X". Sure Heartbleed was bad, but are you going to write your own TLS library? IMO, the OP post has an unfounded sense of hubris. Everyone else's code is bad except for me, who only writes minimal code with no exploits.

> The corollary to "Minimize code ruthlessly" is "Roll your own X". Sure Heartbleed was bad, but are you going to write your own TLS library?

But using "don't roll your own crypto" doesn't generalize to using every possible library to avoid writing any code. Down that path lies the madness of leftpad.

Crypto and security protocol implementations are exceptionally difficult to write in a way that avoids all exploits, so, yeah, don't write your own TLS library unless you're willing to fund a team of all the necessary experts which is going to be very difficult.

OTOH a very large number of libraries implement simple things in overly complex ways (sometimes by necessity because a library needs to try to be all things for all people) and can be very easily replaced by small amounts of code that do only what my product needs.

Every third-party library you add is an additional source of bugs, unmaged surprises (e.g. they decide to break their APIs for the lulz), attack surface and constraints. It is wise to weight the benefits against the drawbacks for each library individually before making it part of your code.

Re: Cull your dependencies

#108

Post by jjav »

wcedmisten wrote: post #4

It would be nice if there were a way to quantify the value of your dependencies. If you have a dependency with 1,000 LoC and your application is utilizing 800 of them, that seems like a good reason to use the dependency. You're (hopefully) getting unit tests, documentation, and public exposure of the code (bugfix opportunities) for "free" If you have a dependency with 1,000,000 LoC and you only need 1,000, that indic…

> This is only a heuristic, but are there any tools that examine metrics like that?

Use a code coverage tool to see what call paths are getting exercised.

Re: Cull your dependencies

#109

Post by jjav »

swatcoder wrote: post #102
smarx007 wrote: post #41

It's a good advice but it has a cost. Where is the discussion about cost? The product with less dependencies will live longer and give you better flexibility but it will cost more to build and more to maintain (incl. onboarding new engineers who need to learn their way around your custom stdlib+). It's a balanced choice but the stakeholders are not prepared to invest more. Furthermore, if the project gets cancelled,…

> will cost more ... to maintain This is a reasonable and popular intuition, but for an enormous class of projects it turns out to be torturously false and a poor reason to choose dependencies. How many times have you sat down on Monday to fix a bug in your product only to discover DoohickyLib 3.3.2 is no longer building correctly with the toolchain update that you just pulled in. So now you go to DoohickyLib's githu…

Very much this. An external dependency gives you the free puppy, but the ongoing work is still yours.

One of the items, as you describe, is that external dependencies introduce unpredictable change on a timeline which is entirely out of your control.

A particularly annoying example that has happened many times is there is an exploit in library A which is now fixed in the latest version so we much upgrade. Oh but the latest version also bumps the dependency of some other library it uses to a version that removed a key feature we need. Infosec says you must fix the vulnerability immediately and of course the product team isn't willing to compromise on the feature loss. Oops. When you own the code you own these decisions.

Of course, some library projects are run very professionally and maintain a strict observance of compatibility within major releases, a long deprecation announcement process and so on. Other library projects, not so much. Definitely favor depending on the first kind and avoid the second kind.

Re: Cull your dependencies

#110

Post by smarx007 »

swatcoder wrote: post #102
smarx007 wrote: post #41

It's a good advice but it has a cost. Where is the discussion about cost? The product with less dependencies will live longer and give you better flexibility but it will cost more to build and more to maintain (incl. onboarding new engineers who need to learn their way around your custom stdlib+). It's a balanced choice but the stakeholders are not prepared to invest more. Furthermore, if the project gets cancelled,…

> will cost more ... to maintain This is a reasonable and popular intuition, but for an enormous class of projects it turns out to be torturously false and a poor reason to choose dependencies. How many times have you sat down on Monday to fix a bug in your product only to discover DoohickyLib 3.3.2 is no longer building correctly with the toolchain update that you just pulled in. So now you go to DoohickyLib's githu…

> but for an enormous class of projects

> with the [latest] toolchain update that you just pulled in

In my experience, larger projects tend to be VERY conservative with toolchain updates. For example, I have Java JDK 8 (2014), 11 (2018), 17 (2021), and 18 (2022) installed; the larger projects are on JDK 11 or are just migrating from JDK 8 to JDK 11. Newer, smaller projects are on JDK 17, and only experimental projects use JDK 18.

> Unfortunately, the reporter isn't experienced at contributing to open source and doesn't want to contribute.

One more reason not to chase bleeding edge but to stay on LTS instead.

Bottom line, I am not removing Google Guava or Apache Jena from my projects because of a few CVEs they may have every few years. I am not sure I will write more secure and maintainable code. And even if I did, would the stakeholder really benefit from that?

Post reply on HN