Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Never use a dependency that you could replace with an afternoon of programming

blog.carlmjohnson.net

Post reply on HN
Print view View original

131–140 of 333 posts

Re: Never use a dependency that you could replace with an afternoon of programming

#131

Post by scarface74 »

VHRanger wrote: post #75
andrewstuart wrote: post #36

Earlier quoted context omitted.

I came here just to say this. "This'll take an afternoon" - three weeks later...... Programmers are notorious for this. BUT even apart from this problem ... you absolutely should use every dependency you can that will save you time. Try to write less code not more. When you write code you write bugs, add complexity, add scope increase need for testing, increase the cognitive load required to comprehend the software,…

Found the NPM user. Dependencies have costs: - Dependencies break over time. They have a nonzero maintenance cost. - They impose API boundaries on you that may not fit your existing data structures - It's harder to change underlying bugs - They might introduce security issues Sure, use dependencies. But there's a reasonable position between "never write any code" and "never take on dependencies". Of which NPM is one…

I could say all of the same things about the in house tools that the “architect” wrote three jobs ago - including the bespoke ORM, object mapper, and logging framework.

Or two jobs ago where two developers who had worked at the company for 10 and 15 years respectively were maintaining a bespoke 15 year old EHR written in PowerBuilder and depended on SQL Server 2003 - in 2016.

Every company thinks they are their own special snowflake where cross cutting concerns can’t be handled by a third party.

Re: Never use a dependency that you could replace with an afternoon of programming

#132

Post by cogman10 »

Ajedi32 wrote: post #108
ghj wrote: post #82

Earlier quoted context omitted.

This is so hilariously overengineered: not failing when passed the wrong types, arbitrarily caching padding of len The most egregious is the bad big O analysis for a pointless binary search. The loop does indeed run O(log(n)) times but `ch += ch` still takes O(ch.length) which is growing exponentially. It ends up being a complicated way of still taking O(n) time while creating a lot of intermediate strings. It isn't…

Did you run benchmarks to validate those assumptions? The developers of left-pad did: https://github.com/left-pad/left-pad/blob/master/perf/perf.j... It's not overengineered if thousands of downstream projects are relying on it, some of which might see significant benefits from those performance optimizations.

Too bad that's a flawed benchmarking methodolgy. JITs are notoriously hard to correctly profile and the benchmark lib isn't even sort of doing the right thing.

For example, it's missing warmup. The results aren't being consumed in a way that wouldn't optimize them away. The framework itself imposes a pretty large amount of overhead (moreso than I'd expect from leftpad).

It is somewhat likely that what they are measuring isn't leftpad performance, but rather how fast the JIT ends up optimizing the benchmark code.

I'd suggest watching this video https://vimeo.com/78900556

It's about microprofiling the JVM, but the principles are the same for other JIT based languages (such as javascript).

Re: Never use a dependency that you could replace with an afternoon of programming

#133

Post by ogre_codes »

gm wrote: post #120
ogre_codes wrote: post #100

Earlier quoted context omitted.

> We think we might only need 1% of an external dependency I've seen the opposite just as often. Someone things an external dependency can just plug in and solve the problem, then 16 hours later they've made the basic thing work with an external dependency with a bunch of internal code on top of it. Now you have a pile of fragile code which breaks when the dependency changes underneath you, you have a bunch of in-hou…

That's exactly what my point is: This is way too complex to boil down to a simple rule of thumb to apply to all cases. There are arguments and counter arguments (and real-life examples and counter examples) ad-nauseum. It's better to say: "Hey experienced people: How would you decide whether to introduce an external dependency or roll your own if the immediate requirement at hand seems small enough to write on your o…

Developers should try to build things before they reach out and grab a package. Usually it doesn't take too long to figure out if you are making a giant hairball. By trying to implement it yourself, you get a better understanding of the problem regardless.

Re: Never use a dependency that you could replace with an afternoon of programming

#134

Post by scarface74 »

TheDong wrote: post #47
axlee wrote: post #28

Earlier quoted context omitted.

Is that really 5 minutes? (For when left-pad was relevant) var cache = [ '', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ' ]; function leftPad (str, len, ch) { // convert `str` to a `string` str = str + ''; // `len` is the `pad`'s length now len = len - str.length; // doesn't need to pad if (len >= 1; // "double" the `ch` so this operation count grows logarithmically on `len` // each time `ch` is "doubled", the `len` w…

One of the points of the article is that when you code a dependency for your purpose, you get smaller code. For example, I know I'll always pass it a string, so I don't need to type-check. I started with the problem of "I need a function that pads a string with a character out to some length". Coding it up took under 1 minute, easily under 5 minutes. function leftPad (str, len, ch) { const neededPadding = len - str.l…

Unless you are doing front end code, why does the size of the code matter?

Re: Never use a dependency that you could replace with an afternoon of programming

#135

Post by earthboundkid »

pacomerh wrote: post #73

nothing takes an afternoon of programming if it needs to be QA'd. At least this is not good advice for a company. Possibly for personal projects.

But you can add a dependency in less than an afternoon?

The exact same objection applies in both cases.

Re: Never use a dependency that you could replace with an afternoon of programming

#136

Post by acdha »

avmich wrote: post #52
acdha wrote: post #22

Earlier quoted context omitted.

That was my first thought: I've seen these projects before — they're where you find 5 slightly different implementations of similar logic, no logging or tests, failures as soon as someone uses Unicode, etc. and I get an order of magnitude performance improvement by replacing that code with an external module which has had the other 19 afternoons' worth of work it actually takes.

> and I get an order of magnitude performance improvement Have you heard the adage about premature optimization being the root of all evil? Yes, even with the second part. What is the premature optimization here, in your opinion? Most of cases developers create something new - that's the state of industry now, not too good but it's how it is. If you'd be refactoring the existing code - sure, find the problem, design…

The point was that when something is large enough to be “an afternoon”, it's probably more work than you're expecting and you haven't yet discovered important details. If there's something which does what you need, it's far more likely that _other people_ have invested time sanding off the rough edges which you have yet to discover.

If it's not hard to use that library you're probably better off unless it's a problem you understand very well and will see a real advantage to tackling differently. For example, if you use a library and don't like it that experience will still be useful for having clarified what exactly it is that you want to do and the rough size of what you're taking on.

Re: Never use a dependency that you could replace with an afternoon of programming

#137

Post by cogman10 »

Ajedi32 wrote: post #123
cogman10 wrote: post #53

Earlier quoted context omitted.

This is actually a great example of why you SHOULDN'T use left pad. I've not profiled it, but I'm going to guess that now-a-days this will be faster than the current implementation on npm. function leftPad (str, len, ch) { str += ''; len = len - str.length; if (len Why? because the VM is (very likely) going to do exactly what the cache would have done. It can replace `ch.repeat(len) + str;` with a presized string all…

They benchmarked it. Their implementation was faster than ch.repeat: https://github.com/left-pad/left-pad/pull/11#issuecomment-20... Nowadays there's a native padStart function and the left-pad package is deprecated as a result.

They used a flawed benchmarking methodology.

JITs only optimize hot code which means any benchmark without a warmup is going to measure cold + hot code time.

Further, JITs will optimize away unused results. They aren't using the leftpad results in any meaningful way.

What they are likely measuring is how long it takes for the JIT to optimize the benchmarking framework.

watch: https://vimeo.com/78900556 for how to microbenchmark a JITed language.

Re: Never use a dependency that you could replace with an afternoon of programming

#138

Post by TheDong »

scarface74 wrote: post #134
TheDong wrote: post #47

Earlier quoted context omitted.

One of the points of the article is that when you code a dependency for your purpose, you get smaller code. For example, I know I'll always pass it a string, so I don't need to type-check. I started with the problem of "I need a function that pads a string with a character out to some length". Coding it up took under 1 minute, easily under 5 minutes. function leftPad (str, len, ch) { const neededPadding = len - str.l…

Unless you are doing front end code, why does the size of the code matter?

"smaller" there was short-hand for "more focused, purpose-built to your problem, less generic".

All of those properties make the code easier to reason about and test.

Being easier to reason about, and being a more exact abstraction for my needs, are both incredibly valuable properties.

Re: Never use a dependency that you could replace with an afternoon of programming

#139

Post by JackFr »

Izkata wrote: post #89
xamuel wrote: post #67

Earlier quoted context omitted.

Well, one special edge-case would be where you only need to parse some extremely tiny subset of JSON (for example: you only need to parse dictionaries whose keys and values are positive integers, like {1:2,3:4}). Then, depending how expensive the full json parser is, it might be worth your while just writing the limited parser yourself. Of course, you might say, inevitably feature-creep will expand the list of things…

Your example is more apt than intended: That's not valid json, which only allows string keys. If you use a library it'll either barf now or later when they fix it, so if you're forced to work with an API like that and can't change it, a custom parser is really the only way to go.

Well the good libraries have option flags that will allow you to handle JSON as found in the wild.

Re: Never use a dependency that you could replace with an afternoon of programming

#140

Post by Ma8ee »

aliceryhl wrote: post #81
xamuel wrote: post #67

Earlier quoted context omitted.

Well, one special edge-case would be where you only need to parse some extremely tiny subset of JSON (for example: you only need to parse dictionaries whose keys and values are positive integers, like {1:2,3:4}). Then, depending how expensive the full json parser is, it might be worth your while just writing the limited parser yourself. Of course, you might say, inevitably feature-creep will expand the list of things…

There are fully correct JSON parsers you aren't going to beat, even if you implement a subset. [1] [1]: https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of...

No, I won't beat them. But if it a limited subset that I can implement with twenty lines straightforward code, that will often be cheaper.

I've been on projects where they imported xml-parsers many times bigger than the rest of the whole codebase just to send a well formatted order number.

Post reply on HN