Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Never use a dependency that you could replace with an afternoon of programming

blog.carlmjohnson.net

Post reply on HN
Print view View original

151–160 of 333 posts

Re: Never use a dependency that you could replace with an afternoon of programming

#151

Post by TheDong »

scarface74 wrote: post #142
TheDong wrote: post #138

Earlier quoted context omitted.

"smaller" there was short-hand for "more focused, purpose-built to your problem, less generic". All of those properties make the code easier to reason about and test. Being easier to reason about, and being a more exact abstraction for my needs, are both incredibly valuable properties.

More generic is better. If I go into a codebase that uses standard libraries. Even if I don’t know how to do something about it someone on the Internet does. Your custom framework - not so much. I don’t have to care about how the underlying libraries work. I can treat them as a black box.

> I don’t have to care about how the underlying libraries work. I can treat them as a black box.

When I wrote haskell, the majority of the libraries did just work, and I didn't have to dig into their code to find bugs often.

When I wrote javascript, hundreds of the libraries I used did not just work. I usually had to care very much about their details because they were poorly implemented, full of bugs and incorrect abstractions, and often abandoned soon after.

I agree that there's benefits in reusing some well-socialized and well-implemented generic frameworks and abstractions. It's not worth using generic abstractions that are not well understood, buggy, and don't match your needs closely. In that case, write your own.

More generic is not always better. Above, I'm arguing that it's important for code to be easier to reason about. If a generic abstraction helps with that, cool, but it's not always going to be the case.

Re: Never use a dependency that you could replace with an afternoon of programming

#152

Post by jkingsbery »

In my 12 year professional career and 10 years of programming before that, I can't count how many times I thought something was an afternoon of programming and I was terribly terribly wrong.

Generally good advice to be conscious about taking dependencies, but it's also worth sanity checking that something is really "an afternoon of programming."

Re: Never use a dependency that you could replace with an afternoon of programming

#153

Post by TeMPOraL »

xamuel wrote: post #67
jdmichal wrote: post #34

My rule is, don't use a dependency to implement your core business. Is JSON parsing our core business? No, so why would we ever write -- and thereby commit to supporting for its entire lifetime -- JSON parsing code? All the code you write and support should be directly tied to what you as a business decide are your fundamental value propositions. Everything else you write is just fat waiting to be cut by someone who…

Well, one special edge-case would be where you only need to parse some extremely tiny subset of JSON (for example: you only need to parse dictionaries whose keys and values are positive integers, like {1:2,3:4}). Then, depending how expensive the full json parser is, it might be worth your while just writing the limited parser yourself. Of course, you might say, inevitably feature-creep will expand the list of things…

I agree.

> Of course, you might say, inevitably feature-creep will expand the list of things your parser needs to parse

If you've done your parser correctly, you'll be able to replace its implementation with the new dependency, with little to no need for extra refactoring in the rest of the codebase.

Re: Never use a dependency that you could replace with an afternoon of programming

#154

Post by jaggederest »

nayuki wrote: post #107
jdmichal wrote: post #34

My rule is, don't use a dependency to implement your core business. Is JSON parsing our core business? No, so why would we ever write -- and thereby commit to supporting for its entire lifetime -- JSON parsing code? All the code you write and support should be directly tied to what you as a business decide are your fundamental value propositions. Everything else you write is just fat waiting to be cut by someone who…

> don't use a dependency to implement your core business In logic language, you're saying "If X is your core business, don't outsource X". > Is JSON parsing our core business? No, so why would we ever write -- and thereby commit to supporting for its entire lifetime -- JSON parsing code? All the code you write and support should be directly tied to what you as a business decide are your fundamental value propositions…

core business = C

outsource = O

Object = x

x ∉ C ↔ O(x)

If the symbols don't show up:

if-and-only-if x is not in C, O(x)

Re: Never use a dependency that you could replace with an afternoon of programming

#155

Post by Ajedi32 »

cogman10 wrote: post #137
Ajedi32 wrote: post #123

Earlier quoted context omitted.

They benchmarked it. Their implementation was faster than ch.repeat: https://github.com/left-pad/left-pad/pull/11#issuecomment-20... Nowadays there's a native padStart function and the left-pad package is deprecated as a result.

They used a flawed benchmarking methodology. JITs only optimize hot code which means any benchmark without a warmup is going to measure cold + hot code time. Further, JITs will optimize away unused results. They aren't using the leftpad results in any meaningful way. What they are likely measuring is how long it takes for the JIT to optimize the benchmarking framework. watch: https://vimeo.com/78900556 for how to mic…

Yeah benchmarks can be much more difficult to get right than it might seem at first glance. Good thing they didn't try to write their own benchmarking code, otherwise they might have fallen into those traps you just mentioned.

Luckily, they didn't, and instead pulled in the the `benchmark` library as a development dependency[1]. The author of said library works on V8, and already considered all those problems and much, much, more[2].

[1]: https://github.com/left-pad/left-pad/blob/master/perf/perf.j...

[2]: https://mathiasbynens.be/notes/javascript-benchmarking

Re: Never use a dependency that you could replace with an afternoon of programming

#156

Post by TeMPOraL »

jkingsbery wrote: post #152

In my 12 year professional career and 10 years of programming before that, I can't count how many times I thought something was an afternoon of programming and I was terribly terribly wrong. Generally good advice to be conscious about taking dependencies, but it's also worth sanity checking that something is really "an afternoon of programming."

Agreed. One has to learn to be realistic, and account for time spent thinking, debugging and testing when coding from scratch. left-pad is an afternoon project. SQLite connector may be a week project. Full JSON parser? Let's book a month to be safe.

Re: Never use a dependency that you could replace with an afternoon of programming

#157

Post by ghj »

Ajedi32 wrote: post #108
ghj wrote: post #82

Earlier quoted context omitted.

This is so hilariously overengineered: not failing when passed the wrong types, arbitrarily caching padding of len The most egregious is the bad big O analysis for a pointless binary search. The loop does indeed run O(log(n)) times but `ch += ch` still takes O(ch.length) which is growing exponentially. It ends up being a complicated way of still taking O(n) time while creating a lot of intermediate strings. It isn't…

Did you run benchmarks to validate those assumptions? The developers of left-pad did: https://github.com/left-pad/left-pad/blob/master/perf/perf.j... It's not overengineered if thousands of downstream projects are relying on it, some of which might see significant benefits from those performance optimizations.

They failed at writing a correct O(n) left pad in their benchmark:

https://github.com/left-pad/left-pad/blob/master/perf/O(n).j...

They are repeatedly appending a single character to the front of a string. This is actually O(n^2) so of course they are winning against it.

Re: Never use a dependency that you could replace with an afternoon of programming

#158

Post by unilynx »

rootusrootus wrote: post #141
hungry_haibt wrote: post #101

Earlier quoted context omitted.

Great rule. I was wondering, how do you manage updating the Jackson JSON parsing package. What if you have 100 such packages and they get updated weekly with breaking changes ?

There's lots of opinions on this, all with good justification. My current team leaves most dependencies unlocked and depends on good automated tests to sniff out broken dependencies. If necessary we lock dependencies to a particular version or range (e.g. Some people just never upgrade until they need to. That's workable, though when you do need to upgrade a package you may be spending the rest of the week working ou…

If you only upgrade when you need to, but not necessarily to the latest versions, odds are that whatever breakage is caused by the latest nodejs/npm/etc incompatibility has already been documented in issue trackers or stackoverflow

Re: Never use a dependency that you could replace with an afternoon of programming

#159

Post by bob1029 »

I agree with this in principle. There is nuance, but the only way I look at 3rd party dependencies is as liabilities, normalizing for function.

The only question for me from this point is "Which dependencies are the biggest liabilities for us?" The answer to this question depends wildly on what exactly it is that you are trying to do.

Overall, our strategy is to typically use a 3rd party dependency by default in order to quickly get a MVP rolled out. Once we have a clearer picture of how we can solve some particular problem in a concrete way, we make a decision regarding whether we should drop that dependency or keep it around. Most of the time, we will develop an interface which exposes our need for the dependency in a vendor-agnostic way. Then, we can target any arbitrary implementation against this interface and quickly swap them around as required in DI.

The biggest question in these discussions is always going to be "Well... how long would it take to write our own?". Even if someone is being realistic and gives you something 2x as long as you were hoping for, you should consider all of the other factors. Keep in mind that if you write it yourself, you can probably iterate on it without much difficulty as well (i.e. custom change requests that a 3rd party would completely ignore). Conversely, if you have to maintain it and its really buggy, you can't hope someone else is going to eventually solve your problems for you.

Re: Never use a dependency that you could replace with an afternoon of programming

#160

Post by munk-a »

hungry_haibt wrote: post #101
jdmichal wrote: post #34

My rule is, don't use a dependency to implement your core business. Is JSON parsing our core business? No, so why would we ever write -- and thereby commit to supporting for its entire lifetime -- JSON parsing code? All the code you write and support should be directly tied to what you as a business decide are your fundamental value propositions. Everything else you write is just fat waiting to be cut by someone who…

Great rule. I was wondering, how do you manage updating the Jackson JSON parsing package. What if you have 100 such packages and they get updated weekly with breaking changes ?

For what reason are you updating your packages? Is there a severe security issue in that package or, if it works today, could you pin it to that version and wait until there is a compelling reason to update it.

Here's some reasoning - if this project was inhoused would we detect and patch it any quicker? Would we have a dev constantly assigned to it that would be pushing out patches to the rest of the team... or is it the sort of software we'd write once and then wait until a compelling reason to invest more into. Whether software is inhouse or outsourced you still retain decision making about how much time to invest in its maintenance.

Post reply on HN