Libraries are under-used. LLMs make this problem worse
1–10 of 59 posts
Re: Libraries are under-used. LLMs make this problem worse
#2I don't know about vibe coding (I'm not a fan of vibe coding) but LLM agents make me more likely to use good libraries, not less, because they instantly know how to use them; there's less intellectual friction to breaking them out (don't have to find and add the dep, don't have to look for the example code). These kinds of things made me ultra-likely to just hand-code crappier versions of stuff libraries did, before I got LLM-assisted.
Re: Libraries are under-used. LLMs make this problem worse
#3”Dunning-Kruger effect leads us to understimate the complexity of the problem solved by the library we're considering.”
Invoking the smarter-than-thou effect is not a great starting point.
See e.g. https://www.sciencedirect.com/science/article/abs/pii/S01602...
If we’re considering a library, it would be prudent of us to take a look at the source code to see what exactly we’re pulling in. In the process, we would learn about the lay of the land, the API and the internals, and get at least an overview of the complexity of the problem it solves.
Re: Libraries are under-used. LLMs make this problem worse
#4I disagree. Every python package we install seems to install dozens of libraries, each of which can could harbour malware. Many of them are only used for a single function within them. We have no idea of what most of the packages are for. It's a lot.
Re: Libraries are under-used. LLMs make this problem worse
#5I dunno, often people say libraries are over used, at least in the JavaScript world.
My celery/RabbitMQ-based web crawler failed because of the Cloudflare CAPTCHAs, I figured it was best to empty out the queue and archive it. I asked copilot what to do and it told me to use a CLI program. “Does that come with RabbitMQ?” “No, you download it from GitHub”. It offered to write me a Python script but the CLI program did exactly what I needed. It got an option wrong but I’d expect the same if I asked a friend for help.
Re: Libraries are under-used. LLMs make this problem worse
#6”Dunning-Kruger effect leads us to understimate the complexity of the problem solved by the library we're considering.” Invoking the smarter-than-thou effect is not a great starting point. See e.g. https://www.sciencedirect.com/science/article/abs/pii/S01602... If we’re considering a library, it would be prudent of us to take a look at the source code to see what exactly we’re pulling in. In the process, we would lea…
I learned to consider that if one brings up Dunning-Kruger...projection/irony may be at play.
Anyways...I've had a few reoccurring issues with libraries. Note that the language is framed on a case by case basis...not general rules.
1. The essential implementation is a small amount of code...wrapped in structures just for packaging essential code. The wrapping code can be larger & more complex than the essential code.
2. There's small differences between what's needed & what's provided. Which requires workarounds for the desired outcome. These workarounds muddy the logic & can be pervasive at scale.
3. There can be dissonance between the app architecture & the library api.
4. Popular libraries in particular...create a culture of thinking in terms of the library/framework. Leading to resource inefficiencies...And outright dismissing solutions that are a better match for the domain. In short, the library/framework api frames the problem & solution...Which may not match the actual problem & optimal solution.
5. The library/framework authors are concerned about promoting the library/framework. Not solving the actual problem. Many problems need to be solved. The library/framework just be the "Golden Hammer" to pound in your screw.
With all that being said...there are many useful libraries that define & solve problems in their particular domain. Particularly with common, well defined, appropriately scoped requirements.
Re: Libraries are under-used. LLMs make this problem worse
#7I don't know if this is true. An LLM just today recommended a library I had never heard of, and after doing some due diligence, it looks solid.
This is analogous to folks who claim nobody is going to be able to learn software engineering any more. I think it is just the opposite. LLMs can be an awesome tool for learning.
Re: Libraries are under-used. LLMs make this problem worse
#8I disagree. Every python package we install seems to install dozens of libraries, each of which can could harbour malware. Many of them are only used for a single function within them. We have no idea of what most of the packages are for. It's a lot.
This. We finally have a tool that can learn from all the libraries and abstractions that have to fit everybody's needs (and do so badly because there is no free lunch), and extract just the parts that are actually relevant to our problem and domain. This allows you to not only produce a much smaller attack surface, but also allows for domain specific optimisations and shortcuts.
It's kinda like project specific semantic monomorphization.
Re: Libraries are under-used. LLMs make this problem worse
#9”Dunning-Kruger effect leads us to understimate the complexity of the problem solved by the library we're considering.” Invoking the smarter-than-thou effect is not a great starting point. See e.g. https://www.sciencedirect.com/science/article/abs/pii/S01602... If we’re considering a library, it would be prudent of us to take a look at the source code to see what exactly we’re pulling in. In the process, we would lea…
The Dunning-Kruger effect absolutely also leads to people releasing libraries they should not have and which nobody should use.
Re: Libraries are under-used. LLMs make this problem worse
#10If your vibe coding prompt generated a 1000 line output, you should probably ask if there is a library that would do that for you. If not, library is not worth it to shorten a one pager.