Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Never use a dependency that you could replace with an afternoon of programming

blog.carlmjohnson.net

Post reply on HN
Print view View original

251–260 of 333 posts

Re: Never use a dependency that you could replace with an afternoon of programming

#251

Post by smabie »

yoz-y wrote: post #124
smabie wrote: post #42

Earlier quoted context omitted.

The best way to avoid dependencies is to use a language that is built in a way such that dependencies are worthless. Like APL, J, or kdb+/q. All of these languages are incredibly small, have almost non-existent standard libraries, and yet are designed in such a way that a large standard library becomes superfluous. Having a large standard library speaks poorly of the composability and orthogonality of language primit…

Sure, if the problems you are solving are well suited for languages like APL, J or kdb+/q...

Call me an iconoclast, but I think array languages are more suitable for general purpose programming than even general purpose languages.

Re: Never use a dependency that you could replace with an afternoon of programming

#252

Post by michaericalribo »

nayuki wrote: post #221
soedirgo wrote: post #193

Earlier quoted context omitted.

Since we're in pedanticville, these aren't converses, but inverses. The converse goes "If you don't outsource X, then X is your core business".

Thanks, you are right. A converse is logically equivalent to an inverse, so I'm only half wrong. =)

Ah, yes, everyone knows the three values of Boolean logic...

Re: Never use a dependency that you could replace with an afternoon of programming

#253

Post by Shank »

nullc wrote: post #229
earthboundkid wrote: post #122

OP here: A lot of people are objecting, "What if you estimate wrong, and it takes more than an afternoon?" This objection is very bad. It is not possible to add a new dependency in less than afternoon because you need to evaluate alternatives, test it, learn how it works, make sure it's not accidentally GPL, etc. So there are not two methods, the less-than-an-afternoon method and the more-than-an-afternoon method. Th…

> But now you know what the dependency is really supposed to do and why it's non-trivial, so you're in an even better position to evaluate which ones are good. I came in here to say this. If you think you're not qualified to write the function, you're probably also equally unqualified to choose someone else's implementation of it. There is a lot of stuff out there-- stuff which is widely used-- which is not fit for y…

Not a lot of people can correctly write cryptography code on the first try, but we definitely advocate for people pulling well known cryptography libraries and using them instead of building their own, for obvious reasons. Not many people are qualified to write a lot of things, but are capable of making sound dependency judgements with heuristics. The trick is to use good heuristics and to not use a library for every tiny thing.

Re: Never use a dependency that you could replace with an afternoon of programming

#254

Post by browsergap »

I like this, but not things I wrote instead of using existing lib took weeks, not an afternoon. And I didn't think they would take an afternoon.

If it's trivial, probably use existing lib.

I don't know what my rule for deciding this is....I don't think I have one. I just make decisions. If I don't like a library I won't use it.

Re: Never use a dependency that you could replace with an afternoon of programming

#255

Post by nullc »

Igelau wrote: post #249
nullc wrote: post #234

Earlier quoted context omitted.

> BRB, going to go roll my own AES -- what could possibly go wrong? Probably not much more than picking an AES library written from someone else. It's fairly unlikely that your custom AES would function at all unless it was functionally correct. Your bespoke AES will likely have timing sidechannels, but so will most AES you go pick off the shelf. (in fact, if you happen to need CBC mode, virtually any AES you pick of…

Yes, yes, yes, and yes, but the crucial difference is that vulnerabilities in the library are more likely to have the benefit of disclosure.

Eh. Yes, more likely, though unless you're talking about something with openssl level ubiquity it might not do much.

For example: my post pointed out that Linux's naive AES has a huge timing sidechannel (e.g. the same bug JoeBob's would). This isn't news. It's also not fixed.

Many times I've been asked to review a cryptographic library and found that it had problems, had them for been for years.. Sometimes the issues had been reported and just ignored.

In some cases reporting the issue just causes the author to take it down... creating its own problems for people who were depending on it!

At the moment I have two private outstanding bug reports for total breaks in cryptosystem library code that I just stumbled into while browsing the internet where the authors/maintainers haven't replied and it's been more than a month. After a bit longer, I'll make the reports in public, but I expect the software will continue to go uncorrected (or just be taken down in response).

One piece of advice I'd give for anyone taking a dependency: go read through its bug tracker of open bugs (and recently resolved ones) -- and their public patch queue if they have one. Also do the same for all transitive dependencies. You can gain some pretty valuable knowledge and more benefit from shared bug finding.

Of course, if you're not a subject matter expert you might not be able to judge if a report is correct or if the subject is serious-- though you will probably be able to tell if the maintainers are active/responsive.

I gave a talk once on the problem of "Selection Cryptography"-- where I argue that merely _picking_ an implementation of cryptographic code (much less the primitives to use) is an act of rolling your own cryptography that triggers similar risks to writing some which also must be managed.

Re: Never use a dependency that you could replace with an afternoon of programming

#256

Post by ChrisMarshallNY »

I should mention that one thing that I always do with dependencies, these days, is encapsulate them.

I use things like Dependency Inversion or Dependency Injection, or simple "glue" APIs.

This is from being burned by integrating dependencies into my projects, and painting myself into corners.

It helps to do things like swap out DB backends, or things like mapping libraries. YAGNI is important, but it's also important to encourage flexibility. Dependency encapsulation is pretty much perfect for that.

Of course, there are probably cases where it's impossible to use some dependencies without making them integral parts of the project, but it's been my experience that I can write lots of stuff, based on modules.

My work is basically a clump of dependencies; it's just that I wrote the dependencies.

I like modular architectures, with each module/layer as an independent-lifecycle project; complete with testing, documentation, and APIs. Takes a bit longer to write the modules, but the integration is quite smooth.

Re: Never use a dependency that you could replace with an afternoon of programming

#257

Post by kiawe_fire »

jeffbee wrote: post #117
sparker72678 wrote: post #14

Probably good advice, but when was the last time a programmer accurately scoped a problem when they said it will take “an afternoon” to build?

This goes both ways. When was the last time someone properly scoped the maintenance effort of an external library? This goes double for external systems, like kafka or mysql. I've never seen anyone so far even get within two orders of magnitude of the real cost of operating kafka, much less an organization that accurately compared that to the cost of DIY.

The "don't reinvent the wheel" argument often acts as though using a 3rd party lib is "free", and building it yourself is costly with no benefit.

This is sometimes true, but often not. From SFTP libraries to SVG rendering libraries, there have probably been about 3-5 major dependencies of my company's project that I have had to learn and extend or fix bugs in to make them work just in the last year.

And sometimes this means using our own fork that we have to keep maintained.

I'm not saying I would have rather written these particular dependencies from scratch, but they were definitely not cost free. Nor are they all of better quality than what I would have produced had I written them from scratch.

That's the other common refrain - to "defer to the expertise of the crowd".

Don't get me wrong, many 3rd party libraries are of great quality by amazing men and women who I am very thankful for. But certainly not all of them.

There's no magic that says "every third party library is made by an expert with the highest standards".

Re: Never use a dependency that you could replace with an afternoon of programming

#258

Post by toomanybeersies »

markstos wrote: post #113
hungry_haibt wrote: post #101

Earlier quoted context omitted.

Great rule. I was wondering, how do you manage updating the Jackson JSON parsing package. What if you have 100 such packages and they get updated weekly with breaking changes ?

Only update dependencies when your code requires the new version, depends on a bug fix or it fixes a security vulnerability. Otherwise, continue using the same version. Have good test coverage to catch bugs that may originate in dependencies and subscribe to a third-party service to track vulnerabilities in your dependencies.

Then you get 5 year out of date packages, which eventually have a security vulnerability, and now you have the task of upgrading and working through 5 years of (potentially) breaking changes and deprecations.

It's generally easier in the long run to keep your dependencies up to date. If a package has a new breaking change each week, that's a sign you probably shouldn't be using it for production code.

Re: Never use a dependency that you could replace with an afternoon of programming

#259

Post by rsanek »

Does it really take people "a few hours" to integrate a dependency? For personal projects it's more like 15 minutes, and even for stuff at work it doesn't generally take longer than an hour.

Broadly I find that FOSS libraries are great when you stick with their happy path. If you have unique/specific requirements, it may make sense to build it yourself. But there's no reason you can't spend an hour trying to use something for free before re-writing from scratch.

Re: Never use a dependency that you could replace with an afternoon of programming

#260

Post by s3cur3 »

earthboundkid wrote: post #122

OP here: A lot of people are objecting, "What if you estimate wrong, and it takes more than an afternoon?" This objection is very bad. It is not possible to add a new dependency in less than afternoon because you need to evaluate alternatives, test it, learn how it works, make sure it's not accidentally GPL, etc. So there are not two methods, the less-than-an-afternoon method and the more-than-an-afternoon method. Th…

So what I'm taking away from this is that for maximum productivity, you just have to:

1. Not evaluate any alternatives

2. Not read the docs

3. Not check the code

4. Not check the license

;)

(Thanks for a great blog post that articulates something I've felt for a long time.)

Post reply on HN