Libraries are under-used. LLMs make this problem worse

Post by unclad5968 » Fri, Jun 20, 2025, 11:19 PM UTC

fmbb wrote: post #9

cluckindan wrote: post #3

”Dunning-Kruger effect leads us to understimate the complexity of the problem solved by the library we're considering.” Invoking the smarter-than-thou effect is not a great starting point. See e.g. https://www.sciencedirect.com/science/article/abs/pii/S01602... If we’re considering a library, it would be prudent of us to take a look at the source code to see what exactly we’re pulling in. In the process, we would lea…

The Dunning-Kruger effect absolutely also leads to people releasing libraries they should not have and which nobody should use.

Post by simonw » Fri, Jun 20, 2025, 11:23 PM UTC

nimish wrote: post #25

simonw wrote: post #24

Earlier quoted context omitted.

That's part of the idea behind https://llmstxt.org/ - even if you ignore the "/llms.txt" URL there's a bunch of thinking around that to help write explanations of things like libraries that can be used to "teach" a model to use it by injecting that into a prompt. I'm still not clear on what the best patterns for this are myself. I've been experimenting with dumping my entire documentation into the model as a single f…

At this point it seems like just learning the library is easier than trying to cram the documentation into an LLM compatible format.

Post by [deleted] » Fri, Jun 20, 2025, 11:33 PM UTC

kianN wrote: post #15

Unrelated: I initially expected this articles to be referring to public libraries. I think that would be a challenging connection to prove despite it making intuitive sense. On the article: some use cases eg handling dates, fault tolerant queues have so many edge cases and are so mission critical that relying on a battle tested tool makes a lot of sense. However, in my career I’ve seen a lot of examples of a package…

Post by Noumenon72 » Fri, Jun 20, 2025, 11:56 PM UTC

handfuloflight wrote: post #31

lazide wrote: post #26

Earlier quoted context omitted.

This is hilarious .

You're right. I didn't fully read what the OP was saying, which is genius; and my response was more towards the article.

  fn identity(x: T) -> T {
      x
  }  

  fn identity_i32(x: i32) -> i32 { x }
  fn identity_string(x: String) -> String { x }

  import pandas
  df = [{"a": 1}, {"a": 2}]
  total = sum(d["a"] for d in df)

  data = [{"a": 1}, {"a": 2}]
  total = sum(d["a"] for d in data)

Post by lazide » Sat, Jun 21, 2025, 2:15 AM UTC

Noumenon72 wrote: post #47

handfuloflight wrote: post #31

Earlier quoted context omitted.

You're right. I didn't fully read what the OP was saying, which is genius; and my response was more towards the article.

I didn't get this either so let me try to explain as ChatGPT did to me: Monomorphization means taking a generic function and generating a version specific to the type being used, eg a Rust function fn identity (x: T) -> T { x } can be compiled into one version for i32 and one for String, which is more efficient since the compiler knows the types: fn identity_i32(x: i32) -> i32 { x } fn identity_string(x: String) -> S…