Tripping over the potholes in too many libraries • Hacker News Board

Re: Tripping over the potholes in too many libraries

#51

Post by marcus_holmes » Tue, Aug 11, 2020, 11:28 AM UTC

mleonhard wrote: post #47

marcus_holmes wrote: post #23

Earlier quoted context omitted.

I think the answer is to not import the dependency at all. If you absolutely can't write a single function, then copy/pasting it from a set of "known good" functions would be better than importing a dependency. Of course, just writing the function would be better. It would probably take less time than discovering the package, working out the interface for it, discovering that it doesn't actually work for your use cas…

Many useful libraries cannot be easily re-created. Consider datetime handling and X.509 parsing.

I agree, and happily import moment.js for that reason. But that's not what the GP was talking about. This is not a "microdependency". It's something that should have been included in the standard library, except that JS doesn't have one of those.

Re: Tripping over the potholes in too many libraries

#52

Post by yellowapple » Tue, Aug 11, 2020, 7:58 PM UTC

marcus_holmes wrote: post #50

habitue wrote: post #44

Earlier quoted context omitted.

> But once they've done it your junior devs will know how to write a config handling library This is true, but there are always so many things that need building. It's kind of like needing an end table, and either buying one from Target or building it yourself in your garage. End tables aren't that hard to build! The end table at Target is a piece of crap! You'll learn a lot trying to build your own end table! These…

> It's kind of like needing an end table, and either buying one from Target or building it yourself in your garage. Except that, in this analogy, you're a carpenter. Building furniture is your profession, and hopefully something you enjoy doing. Learning how to build end tables quickly is a useful skill to have in your profession. Practising that skill by building an end table for yourself is not a waste of time. > I…

> Except that, in this analogy, you're a carpenter.

Alright, and if you're a junior carpenter, are you going to be tasked with chopping down the perfect trees and whittling those down into the perfect legs and tabletops from which you will assemble your perfect end tables? Probably not; you're almost certainly gonna start off by, at most, going to your local hardware store, buying whatever shitty wood they've got, and putting that together into something that might look god-awful but at least functions approximately like an end table. Only after you've gotten good at building end tables with off-the-shelf wood are you likely going to have the requisite understanding of the limitations of said off-the-shelf wood to have any interest (let alone understanding) in going with custom wood.

Similar deal with software. If you're a junior programmer, yeah, you might be encouraged to develop your skills by developing libraries, if you're lucky, but in most shops you're probably gonna be tasked with, you know, actually delivering software, and that software will probably need to use existing libraries for expediency's sake. It's exactly why a hefty chunk of web development nowadays involves full-blown frameworks like Rails or Django (or even lighter-weight alternatives like Sinatra and Flask, respectively) instead of people hand-crafting HTTP request parsers / response generators and hooking 'em up to raw sockets.

"Store-bought" libraries, like store-bought wood, ain't perfect by any stretch. They still provide a reasonable starting point for getting a project done reasonably quickly instead of getting stuck in the weeds (literally) of growing your own trees.

Re: Tripping over the potholes in too many libraries

#53

Post by marcus_holmes » Wed, Aug 12, 2020, 11:11 AM UTC

yellowapple wrote: post #52

marcus_holmes wrote: post #50

Earlier quoted context omitted.

> It's kind of like needing an end table, and either buying one from Target or building it yourself in your garage. Except that, in this analogy, you're a carpenter. Building furniture is your profession, and hopefully something you enjoy doing. Learning how to build end tables quickly is a useful skill to have in your profession. Practising that skill by building an end table for yourself is not a waste of time. > I…

> Except that, in this analogy, you're a carpenter. Alright, and if you're a junior carpenter, are you going to be tasked with chopping down the perfect trees and whittling those down into the perfect legs and tabletops from which you will assemble your perfect end tables? Probably not; you're almost certainly gonna start off by, at most, going to your local hardware store, buying whatever shitty wood they've got, an…

I think we're probably extending the analogy too far, but I'll play along.

We don't craft programs out of machine code any more. We use high-level languages. So that's where we're not growing our own trees any more. I think the rest of your analogy agrees with me: even junior carpenters need to know how to build an end table by actually building them rather than buying them from Target.

It's a judgement call, I'll agree. But only allowing your junior devs to write glue code for imported dependencies means that's all they know how to do. They'll get the idea that that's all coding is - find the right dependency and write some glue code, and you're done.

I've seen this attitude so much recently, especially in startup product development, and it's actually dangerous. No-one audits their dependecies, they just read the description, import it and try to work out how to make it do the thing. If it works, who cares if it's huge, or has 34539473 other dependencies, or has been taken over by a malicious maintainer? Velocity, right? Get it into production and on to the next feature.

This is going to end badly.

Re: Tripping over the potholes in too many libraries

#54

Post by yellowapple » Wed, Aug 12, 2020, 10:36 PM UTC

marcus_holmes wrote: post #53

yellowapple wrote: post #52

Earlier quoted context omitted.

> Except that, in this analogy, you're a carpenter. Alright, and if you're a junior carpenter, are you going to be tasked with chopping down the perfect trees and whittling those down into the perfect legs and tabletops from which you will assemble your perfect end tables? Probably not; you're almost certainly gonna start off by, at most, going to your local hardware store, buying whatever shitty wood they've got, an…

I think we're probably extending the analogy too far, but I'll play along. We don't craft programs out of machine code any more. We use high-level languages. So that's where we're not growing our own trees any more. I think the rest of your analogy agrees with me: even junior carpenters need to know how to build an end table by actually building them rather than buying them from Target. It's a judgement call, I'll ag…

> We don't craft programs out of machine code any more. We use high-level languages.

A.k.a. we chop down existing trees instead of growing our own.

> This is going to end badly.

While I don't disagree, the alternative is more often than not for nothing to ever get shipped because the development team is busy reinventing the universe, especially when a "don't use libraries" mentality gets taken to its logical conclusions ("we need perfect custom hardware to run our perfect custom operating system written to run our perfect custom programming language/runtime to run our perfect application" - just because Google has those sorts of resources doesn't mean a lone developer trying to get a side project done has those sorts of resources).

There is, however, a reasonable middle ground: start with those dependencies, and then as you encounter their limitations start working to replace them. The vast majority of development teams don't have the resources to make "do everything in-house" the default, but they might have the resources to selectively fork or replace a critical library once they've built an initial implementation of their project on top of said library. To your point about junior devs and learning opportunities, this is a perfect way to learn: "alright, so you built this app with this library, great, now write a replacement for that library that does A, B, and C instead of X, Y, and Z".

Re: Tripping over the potholes in too many libraries

#55

Post by manicdee » Thu, Aug 13, 2020, 12:12 AM UTC

tarkin2 wrote: post #16

manicdee wrote: post #13

Earlier quoted context omitted.

What about git submodules?

Oh lord no. They’re a pain to manage. The world needs fewer gitsubmodules.

How do you go about preparing a PR for an upstream if all the code is magically in your own version control system?

And then how do you separate site-specific patches from bug fixes and new features for the upstream?

Re: Tripping over the potholes in too many libraries

#56

Post by marcus_holmes » Thu, Aug 13, 2020, 12:09 PM UTC

yellowapple wrote: post #54

marcus_holmes wrote: post #53

Earlier quoted context omitted.

I think we're probably extending the analogy too far, but I'll play along. We don't craft programs out of machine code any more. We use high-level languages. So that's where we're not growing our own trees any more. I think the rest of your analogy agrees with me: even junior carpenters need to know how to build an end table by actually building them rather than buying them from Target. It's a judgement call, I'll ag…

> We don't craft programs out of machine code any more. We use high-level languages. A.k.a. we chop down existing trees instead of growing our own. > This is going to end badly. While I don't disagree, the alternative is more often than not for nothing to ever get shipped because the development team is busy reinventing the universe, especially when a "don't use libraries" mentality gets taken to its logical conclusi…

I agree with that. I think I just draw the line further down the scale than you. Possibly, that's because I'm an old fart and more used to having to write everything from scratch, and so therefore more used to it (and less liable to consider that wasted effort).

Re: Tripping over the potholes in too many libraries

#57

Post by kelnos » Mon, Aug 17, 2020, 8:44 AM UTC

marcus_holmes wrote: post #27

kelnos wrote: post #14

> My guess was that people get into these situations where it seems like a library is going to be a solid "100% solution", and yet it lets you down and maybe reaches the 80% mark. That's probably true. I think a difficulty in avoiding this is that often the person reaching for this library, if they tried to write it themselves, would only hit the 60% mark. And I don't think there's anything wrong with that. There are…

I'm always amazed by this attitude. That somehow the authors of random libraries on npm are amazing coders whose code quality can't be touched by mere mortals like us. Firstly, if you've always imported a dependency to get around a problem, then yes it's going to be hard to solve that problem yourself. But it's also a learning experience. Keep doing it and eventually it won't be hard. Secondly, the library probably i…

> That somehow the authors of random libraries on npm are amazing coders whose code quality can't be touched by mere mortals like us.

That's not what I said, nor what I believe. But I think it absolutely is true that most developers would struggle to reproduce the functionality of a fairly meaty dependency they've chosen (that is, not most of the dreck on the npm registry).

If a dependency exists for what you need to do, and after some vetting believe it to be of good quality, it's highly unlikely that it would be a good use of your employer's time to reimplement it yourself. I agree that it'd probably be a good learning experience to do it, but that's not the only consideration.

As for the rest, there's nothing that you're saying that's particularly wrong, but it's absolutely wrong if you apply it 100% of the time, because that would mean everyone would have their own bespoke web framework, networking library, UI toolkit, etc. I don't expect that you're actually making that argument, so it's a bit uncharitable of you to suggest that I'm making the exact opposite one, that no one should ever think about implementing something themselves.

Dependencies have a cost. Not using a dependency also has a cost. Figuring out which cost is lower is part of the job, and that decision needs to be made several times on every project.