Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tripping over the potholes in too many libraries

rachelbythebay.com

Post reply on HN
Print view View original

11–20 of 57 posts

Re: Tripping over the potholes in too many libraries

#11

Post by KSteffensen »

Ar-Curunir wrote: post #10
habitue wrote: post #3

This comes up all the time and I never understand this attitude. Yes: dependencies are bad. Not having dependencies and writing everything yourself: also bad. Honestly, you have to rely on heuristics in your deps. How active is the project? How simple is the thing it's doing (simple enough to probably not have major bugs, but not so simple it's faster to code it yourself) etc. You get so much velocity from depending…

And by re-implementing a library, you're just importing dependencies into your code, with the difference that now you're responsible for bug fixes and security patches, without the opportunity to inherit performance improvements contributed by others.

You also have a much easier route to fixing issues, since there will be far less politics involved. Probably you will also have a deeper understanding of the code since you wrote it.

Re: Tripping over the potholes in too many libraries

#12

Post by consolenaut »

KSteffensen wrote: post #11
Ar-Curunir wrote: post #10

Earlier quoted context omitted.

And by re-implementing a library, you're just importing dependencies into your code, with the difference that now you're responsible for bug fixes and security patches, without the opportunity to inherit performance improvements contributed by others.

You also have a much easier route to fixing issues, since there will be far less politics involved. Probably you will also have a deeper understanding of the code since you wrote it.

This is true, but I've found it far more common that a predecessor at the company I'm at wrote the code and isn't around to support it anymore. For simple components it's no biggie, but for non-trivial components it can take about the same time to debug as an external module, the benefit of the external module generally being better documentation, although thats not a given.

Re: Tripping over the potholes in too many libraries

#13

Post by manicdee »

vbsteven wrote: post #4

I have been thinking a lot lately about a possible solution for a small portion of this problem: Microdependencies. I'll explain in more detail in the context of JS, but it applies to other languages as well. Currently package repositories like NPM host 2 types of dependencies: big community packages (frameworks, database drivers, validation libraries, query builders,...) and smaller function-scoped utility packages…

What about git submodules?

Re: Tripping over the potholes in too many libraries

#14

Post by kelnos »

> My guess was that people get into these situations where it seems like a library is going to be a solid "100% solution", and yet it lets you down and maybe reaches the 80% mark.

That's probably true. I think a difficulty in avoiding this is that often the person reaching for this library, if they tried to write it themselves, would only hit the 60% mark.

And I don't think there's anything wrong with that. There are people out there writing code at many, many different skill levels. Libraries -- even those that are less than perfect -- let people get things done that they might not be able to do on their own, or would do a worse job of without the library.

Re: Tripping over the potholes in too many libraries

#15

Post by sneak »

I think the generally accepted fix here (despite Rachel’s aversion) is to submit a PR to the file writing library that fixes the corruption issue (likely using atomic rename), then get the tool to bump the version of their dep or vendor in the fixed version.

I’ll admit, though, that the balkanization of code adds overhead from the abstraction. I just don’t think it’s a bad thing, because it’s all very new and things are still shaking out.

Imagine if the fix lands in the config file writer library and all the downstreams regularly upgrade their deps; the fix is now a lot more widespread. This is better than every single end dev knowing about atomic renames, I think.

Re: Tripping over the potholes in too many libraries

#16

Post by tarkin2 »

manicdee wrote: post #13
vbsteven wrote: post #4

I have been thinking a lot lately about a possible solution for a small portion of this problem: Microdependencies. I'll explain in more detail in the context of JS, but it applies to other languages as well. Currently package repositories like NPM host 2 types of dependencies: big community packages (frameworks, database drivers, validation libraries, query builders,...) and smaller function-scoped utility packages…

What about git submodules?

Oh lord no. They’re a pain to manage. The world needs fewer gitsubmodules.

Re: Tripping over the potholes in too many libraries

#17

Post by _y5hn »

Honestly, you have this problem because you put up with it, or are "forced" to use certain software. Why is it everytime I venture into Python, interesting language aside, there are always multitudes of dependency problems even tied to the OS? People put up with it. Choosing quality over fast-food is a choice. Languages such as golang has largely solved much of this while still being pretty portable, and devs are encouraged to minimize deps.

Of course fast-food is tempting and may taste good, but after a while of abstention, this fades away. But it's a choice, and one might miss some opportunities.

Re: Tripping over the potholes in too many libraries

#18

Post by armada651 »

KSteffensen wrote: post #11
Ar-Curunir wrote: post #10

Earlier quoted context omitted.

And by re-implementing a library, you're just importing dependencies into your code, with the difference that now you're responsible for bug fixes and security patches, without the opportunity to inherit performance improvements contributed by others.

You also have a much easier route to fixing issues, since there will be far less politics involved. Probably you will also have a deeper understanding of the code since you wrote it.

If you find the fix yourself you don't need to deal with politics if you don't want to. Submitting pull requests is optional after all.

Re: Tripping over the potholes in too many libraries

#19

Post by rini17 »

sneak wrote: post #15

I think the generally accepted fix here (despite Rachel’s aversion) is to submit a PR to the file writing library that fixes the corruption issue (likely using atomic rename), then get the tool to bump the version of their dep or vendor in the fixed version. I’ll admit, though, that the balkanization of code adds overhead from the abstraction. I just don’t think it’s a bad thing, because it’s all very new and things…

It is likely that fix would break somebody else's code which unwittingly depends on the bug. Then, burden of educating the users would fall on maintainers. Who, most likely, aren't having any of that. (Author even linked a article about this.)

Re: Tripping over the potholes in too many libraries

#20

Post by jankotek »

pronik wrote: post #9

From personal experience: it doesn't take a FAANG type of scale to reveal those "potholes". For most open-source projects (and especially things like pip, npm and other infrastructure and build tools) a simple, almost classical enterprise outbound proxy with authentication and MitM-style HTTPS-reencryption is more than enough kill almost every assumption they have in their code. In my case, that proxy tended to deliv…

Could you explain a bit more? I think most pkg repos are just static directories. If you do https reencryption, and https certs do not match EVERYTHING will break.
Post reply on HN