Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Never use a dependency that you could replace with an afternoon of programming

blog.carlmjohnson.net

Post reply on HN
Print view View original

141–150 of 333 posts

Re: Never use a dependency that you could replace with an afternoon of programming

#141

Post by rootusrootus »

hungry_haibt wrote: post #101
jdmichal wrote: post #34

My rule is, don't use a dependency to implement your core business. Is JSON parsing our core business? No, so why would we ever write -- and thereby commit to supporting for its entire lifetime -- JSON parsing code? All the code you write and support should be directly tied to what you as a business decide are your fundamental value propositions. Everything else you write is just fat waiting to be cut by someone who…

Great rule. I was wondering, how do you manage updating the Jackson JSON parsing package. What if you have 100 such packages and they get updated weekly with breaking changes ?

There's lots of opinions on this, all with good justification. My current team leaves most dependencies unlocked and depends on good automated tests to sniff out broken dependencies. If necessary we lock dependencies to a particular version or range (e.g. Some people just never upgrade until they need to. That's workable, though when you do need to upgrade a package you may be spending the rest of the week working out a cascade of breaking changes.

Re: Never use a dependency that you could replace with an afternoon of programming

#142

Post by scarface74 »

TheDong wrote: post #138
scarface74 wrote: post #134

Earlier quoted context omitted.

Unless you are doing front end code, why does the size of the code matter?

"smaller" there was short-hand for "more focused, purpose-built to your problem, less generic". All of those properties make the code easier to reason about and test. Being easier to reason about, and being a more exact abstraction for my needs, are both incredibly valuable properties.

More generic is better. If I go into a codebase that uses standard libraries. Even if I don’t know how to do something about it someone on the Internet does. Your custom framework - not so much.

I don’t have to care about how the underlying libraries work. I can treat them as a black box.

Re: Never use a dependency that you could replace with an afternoon of programming

#143

Post by greggman3 »

sparker72678 wrote: post #14

Probably good advice, but when was the last time a programmer accurately scoped a problem when they said it will take “an afternoon” to build?

It really depends (haha) on what it is. I needed to copy a file in npm scripts. can't use `cp` because that fails on windows. I looked on npm to copy a file. First hit 197 dependencies, 1170 files, 47000 lines of JavaScript.

all I needed was

    const fs = require('fs');
    fs.copyFileSync(process.argv(2), process.args[3]);
Taking 197 dependencies means 197 things that need updates several times a year at a minimum. Any of those updates could break my code, introduce a bug, add a vulnerability on top of the ones already in the packages. So it's not like adding more dependencies is magically free.

Re: Never use a dependency that you could replace with an afternoon of programming

#144

Post by JackFr »

scarface74 wrote: post #131
VHRanger wrote: post #75

Earlier quoted context omitted.

Found the NPM user. Dependencies have costs: - Dependencies break over time. They have a nonzero maintenance cost. - They impose API boundaries on you that may not fit your existing data structures - It's harder to change underlying bugs - They might introduce security issues Sure, use dependencies. But there's a reasonable position between "never write any code" and "never take on dependencies". Of which NPM is one…

I could say all of the same things about the in house tools that the “architect” wrote three jobs ago - including the bespoke ORM, object mapper, and logging framework. Or two jobs ago where two developers who had worked at the company for 10 and 15 years respectively were maintaining a bespoke 15 year old EHR written in PowerBuilder and depended on SQL Server 2003 - in 2016. Every company thinks they are their own s…

I’ve worked with dozens of guys who loved to reinvent config and logging frameworks, because ... I don’t know why.

I suppose because it was easier than working in the actual problem domain, which they knew little about and didn’t care to learn.

Re: Never use a dependency that you could replace with an afternoon of programming

#146

Post by Floegipoky »

hungry_haibt wrote: post #101
jdmichal wrote: post #34

My rule is, don't use a dependency to implement your core business. Is JSON parsing our core business? No, so why would we ever write -- and thereby commit to supporting for its entire lifetime -- JSON parsing code? All the code you write and support should be directly tied to what you as a business decide are your fundamental value propositions. Everything else you write is just fat waiting to be cut by someone who…

Great rule. I was wondering, how do you manage updating the Jackson JSON parsing package. What if you have 100 such packages and they get updated weekly with breaking changes ?

> What if you have 100 such packages and they get updated weekly with breaking changes?

The solution to that is simple, stop using node.js ;)

Re: Never use a dependency that you could replace with an afternoon of programming

#147

Post by Ajedi32 »

cogman10 wrote: post #132
Ajedi32 wrote: post #108

Earlier quoted context omitted.

Did you run benchmarks to validate those assumptions? The developers of left-pad did: https://github.com/left-pad/left-pad/blob/master/perf/perf.j... It's not overengineered if thousands of downstream projects are relying on it, some of which might see significant benefits from those performance optimizations.

Too bad that's a flawed benchmarking methodolgy. JITs are notoriously hard to correctly profile and the benchmark lib isn't even sort of doing the right thing. For example, it's missing warmup. The results aren't being consumed in a way that wouldn't optimize them away. The framework itself imposes a pretty large amount of overhead (moreso than I'd expect from leftpad). It is somewhat likely that what they are measur…

Yeah benchmarks can be much more difficult to get right than it might seem at first glance. Good thing they didn't try to write their own benchmarking code, otherwise they might have fallen into those traps you just mentioned.

Luckily, they didn't, and instead pulled in the the `benchmark` library as a development dependency. The author of said library works on V8, and already considered all those problems and much, much, more[1].

[1]: https://mathiasbynens.be/notes/javascript-benchmarking

Re: Never use a dependency that you could replace with an afternoon of programming

#148

Post by dogma1138 »

Ma8ee wrote: post #140
aliceryhl wrote: post #81

Earlier quoted context omitted.

There are fully correct JSON parsers you aren't going to beat, even if you implement a subset. [1] [1]: https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of...

No, I won't beat them. But if it a limited subset that I can implement with twenty lines straightforward code, that will often be cheaper. I've been on projects where they imported xml-parsers many times bigger than the rest of the whole codebase just to send a well formatted order number.

And what happens when your parsing needs to be expanded?

Re: Never use a dependency that you could replace with an afternoon of programming

#149

Post by Ma8ee »

scarface74 wrote: post #134
TheDong wrote: post #47

Earlier quoted context omitted.

One of the points of the article is that when you code a dependency for your purpose, you get smaller code. For example, I know I'll always pass it a string, so I don't need to type-check. I started with the problem of "I need a function that pads a string with a character out to some length". Coding it up took under 1 minute, easily under 5 minutes. function leftPad (str, len, ch) { const neededPadding = len - str.l…

Unless you are doing front end code, why does the size of the code matter?

Less code is (in general, to certain limits) easier to understand and requires less maintenance.

Re: Never use a dependency that you could replace with an afternoon of programming

#150

Post by tnorthcutt »

aliceryhl wrote: post #81
xamuel wrote: post #67

Earlier quoted context omitted.

Well, one special edge-case would be where you only need to parse some extremely tiny subset of JSON (for example: you only need to parse dictionaries whose keys and values are positive integers, like {1:2,3:4}). Then, depending how expensive the full json parser is, it might be worth your while just writing the limited parser yourself. Of course, you might say, inevitably feature-creep will expand the list of things…

There are fully correct JSON parsers you aren't going to beat, even if you implement a subset. [1] [1]: https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of...

Indeed: https://github.com/qntm/fastjson
Post reply on HN