Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How well do coding agents use your library?

stackbench.ai

Post reply on HN
Print view View original

31–40 of 62 posts

Re: How well do coding agents use your library?

#31

Post by tantalor »

EGreg wrote: post #15
weitendorf wrote: post #13

We’ve been working on this problem off and on for over a year now. Many models bake knowledge of particular tools/libraries/patterns into their weights very well and others quite poorly. In my experience Claude is quite good at integrating the dog.ceo API and noticeably ignorant when it comes to Postgres features, and it knows gcloud commands enough to very confidently and consistently hallucinate arguments. We’ve ba…

I made a provisional patent this year, about how exactly I would solve this problem. Imagine hiring a "team of developers" who can learn your library and iterate 24/7, improving things, doing support, even letting the pointy-haired boss turn his ideas into reality in a forked sandbox on the weekend. For the last 15 years I've been writing against software patents, and producing open source software that cost me about…

Boooo software patents.

Re: How well do coding agents use your library?

#32

Post by EGreg »

weitendorf wrote: post #17
EGreg wrote: post #15

Earlier quoted context omitted.

I made a provisional patent this year, about how exactly I would solve this problem. Imagine hiring a "team of developers" who can learn your library and iterate 24/7, improving things, doing support, even letting the pointy-haired boss turn his ideas into reality in a forked sandbox on the weekend. For the last 15 years I've been writing against software patents, and producing open source software that cost me about…

I hope we don’t have to challenge it! We’re trying to build a similar kind of experience but for both “sides” of the problem: software provider and software users/integrators.

I guess that's why patents are annoying. I have been Mr. Open Source and against intellectual property for most of the past 15 years. But with AI companies rampantly taking everyone's work and repurposing it, and with VC companies not being very eager to invest in open source, I'm taking a different tack with my AI ventures.

My first two companies are radically open source, and no one cared:

https://github.com/Qbix

https://github.com/Intercoin

And this is what we're doing now with AI, but it's not going to be as open: https://engageusers.ai/deck.pdf

Don't worry, we're not looking to get into it with some random other projects. It's mostly to protect our business model against the Big Tech and enterprises.

Re: How well do coding agents use your library?

#33

Post by dingnuts »

EGreg wrote: post #15
weitendorf wrote: post #13

We’ve been working on this problem off and on for over a year now. Many models bake knowledge of particular tools/libraries/patterns into their weights very well and others quite poorly. In my experience Claude is quite good at integrating the dog.ceo API and noticeably ignorant when it comes to Postgres features, and it knows gcloud commands enough to very confidently and consistently hallucinate arguments. We’ve ba…

I made a provisional patent this year, about how exactly I would solve this problem. Imagine hiring a "team of developers" who can learn your library and iterate 24/7, improving things, doing support, even letting the pointy-haired boss turn his ideas into reality in a forked sandbox on the weekend. For the last 15 years I've been writing against software patents, and producing open source software that cost me about…

don't worry everybody, this guy's profile shows he was a blockchain booster five minutes ago, just another grifter, nothin to see here

Re: How well do coding agents use your library?

#34

Post by imtringued »

9rx wrote: post #23
weitendorf wrote: post #21

Earlier quoted context omitted.

Why did my engineering team handle payments through Stripe instead of building a custom payment processor? Aren’t they supposed to be engineering things?

Coding agents presumably don't know how to deal with non-coding things. Stripe's real value isn't in its technology, but it sorting out the complex human problems associated with payment processing. Sending a number over a network is not any great feat. Getting humans to agree that number has meaning is another matter.

This is an extremely roundabout way of saying that you need a payment processor license.

Re: How well do coding agents use your library?

#35

Post by weitendorf »

EGreg wrote: post #32
weitendorf wrote: post #17

Earlier quoted context omitted.

I hope we don’t have to challenge it! We’re trying to build a similar kind of experience but for both “sides” of the problem: software provider and software users/integrators.

I guess that's why patents are annoying. I have been Mr. Open Source and against intellectual property for most of the past 15 years. But with AI companies rampantly taking everyone's work and repurposing it, and with VC companies not being very eager to invest in open source, I'm taking a different tack with my AI ventures. My first two companies are radically open source, and no one cared: https://github.com/Qbix h…

I think I gave you product feedback on Qbix at some point in the past. I also know several founders who’ve secured funding for open source products and built successful businesses off of them. Open-core is pretty popular out here in the Bay Area.

One thing I’ve learned since staring a company is that early on, your greatest asset is trust in your founder/brand, because it’s the only reason for someone to pay you for something until you get your shit together. I’ve personally had a hard time noticing it in myself sometimes, but I think it’s easy to overlook how outward signaling that might look like distrust (eg making users sign NDAs) damages your own ability to build trust. Since early startups tend to be considered untrustworthy by default it can be really counterproductive. Anyway, I appreciate your non-aggression policy

Re: How well do coding agents use your library?

#36

Post by EGreg »

dingnuts wrote: post #33
EGreg wrote: post #15

Earlier quoted context omitted.

I made a provisional patent this year, about how exactly I would solve this problem. Imagine hiring a "team of developers" who can learn your library and iterate 24/7, improving things, doing support, even letting the pointy-haired boss turn his ideas into reality in a forked sandbox on the weekend. For the last 15 years I've been writing against software patents, and producing open source software that cost me about…

don't worry everybody, this guy's profile shows he was a blockchain booster five minutes ago, just another grifter, nothin to see here

I'd actually consider your criticism seriously, if it was anything other than the usual HN "saw the word blockchain, did an immediate TDLR with the word grift" regardless of what was done or built.

If you had anything substantive to back up what you're saying, we could discuss it, but since you don't... well, I'm actually disappointed but w/e.

Re: How well do coding agents use your library?

#37

Post by EGreg »

weitendorf wrote: post #35
EGreg wrote: post #32

Earlier quoted context omitted.

I guess that's why patents are annoying. I have been Mr. Open Source and against intellectual property for most of the past 15 years. But with AI companies rampantly taking everyone's work and repurposing it, and with VC companies not being very eager to invest in open source, I'm taking a different tack with my AI ventures. My first two companies are radically open source, and no one cared: https://github.com/Qbix h…

I think I gave you product feedback on Qbix at some point in the past. I also know several founders who’ve secured funding for open source products and built successful businesses off of them. Open-core is pretty popular out here in the Bay Area. One thing I’ve learned since staring a company is that early on, your greatest asset is trust in your founder/brand, because it’s the only reason for someone to pay you for…

Would you consider arranging a call to discuss our respective projects? If you're building something along these lines, then I think we might end up joining forces.

I've always preferred collaboration and joining forces building on each other's work, than competition and incompatibility.

https://calendly.com/engageusers/meeting

Re: How well do coding agents use your library?

#38

Post by richardblythman »

bdhcuidbebe wrote: post #8
richardblythman wrote: post #2

If coding agents are the new entry point to your library, how sure are you that they’re using it well? I asked this question to about 50 library maintainers and dev tool builders, and the majority didn't really know. Existing code generation benchmarks focus mainly on self-contained code snippets and compare models not agents. Almost none focus on library-specific generation. So we built a simple app to test how well…

> If coding agents are the new entry point to your library, how sure are you that they’re using it well? > I asked this question to about 50 library maintainers and dev tool builders, and the majority didn't really know. Why should they even bother to answer such a loaded and hypothetical question?

im paraphrasing. the questions i asked to dev tool builders were more neutral.

Re: How well do coding agents use your library?

#39

Post by richardblythman »

grim_io wrote: post #29
richardblythman wrote: post #26

Earlier quoted context omitted.

we send out an email when the tests are finished (takes about 30 mins)

That makes you sound like you are dodging the question.

i mean that we wanted an email address to send the results to when they finish.

based on comments here, i do think we should allow users to run the audit first (and provide an email address if they want us to follow up with results later).

Post reply on HN