61–62 of 62 posts
I can foresee it being useful when I write documentation, e.g. given an existing repo and my new guide, can the AI agent implement said feature? Does it fumble?
I only got one run it looks like for free and it was successful so I can't see what would happen if it failed. Curious what sort of feedback it provides on failure.
We’ve been working on this problem off and on for over a year now. Many models bake knowledge of particular tools/libraries/patterns into their weights very well and others quite poorly. In my experience Claude is quite good at integrating the dog.ceo API and noticeably ignorant when it comes to Postgres features, and it knows gcloud commands enough to very confidently and consistently hallucinate arguments. We’ve ba…
I made a provisional patent this year, about how exactly I would solve this problem. Imagine hiring a "team of developers" who can learn your library and iterate 24/7, improving things, doing support, even letting the pointy-haired boss turn his ideas into reality in a forked sandbox on the weekend. For the last 15 years I've been writing against software patents, and producing open source software that cost me about…