Live data from Hacker News

Generative AI coding tools and agents do not work for me

blog.miguelgrinberg.com

211–220 of 464 posts

Re: Generative AI coding tools and agents do not work for me

#211
post #209

I use AI daily, currently paying for Claude Code, Gemini and Cursor. It really helps me on my personal toy projects, it’s amazing at getting a POC running and validate my ideas. My company just had internal models that were mediocre at best, but at the beginning this year they finally enabled Copilot for everyone. At the beginning I was really excited for it, but it’s absolutely useless for work. It just doesn’t work…

I think a first step for these big enterprise codebases (also applicable to documentation) is to collect it into a big ball and finetune on it.

Re: Generative AI coding tools and agents do not work for me

#212

> Another common argument I've heard is that Generative AI is helpful when you need to write code in a language or technology you are not familiar with. To me this also makes little sense. I'm not sure I get this one. When I'm learning new tech I almost always have questions. I used to google them. If I couldn't find an answer I might try posting on stack overflow. Sometimes as I'm typing the question their search wo…

So AI is basically best as a search engine.

I mean, it's just a compressed database with a weird query engine.

Re: Generative AI coding tools and agents do not work for me

#213

Earlier quoted context omitted.

Yes. More things should be rote recitations . Rote code is easy to follow and maintain. We get in trouble trying to be clever (or DRY) --- especially when we do it too early. Important tangential note: the eBPF verifier doesn't "cut out the chaff". It rejects good, valid programs . It does not care that the programs are valid or good; it cares that it is not smart enough to understand them; that's all that matters. T…

> We get in trouble trying to be clever (or DRY) Certainly, however: > That's the point I'm making about reviewing LLM code: you are not on the hook for making it work The second portion of your statement is either confusing (something unsaid) or untrue (you are still ultimately on the hook). Agentic AI is just yet another, as you put it way to "get in trouble trying to be clever". My previous point stands - if it wa…

> The second portion of your statement is either confusing (something unsaid) or untrue (you are still ultimately on the hook).

You're missing the point.

tptacek is saying he isn't the one who needs to fix the issue because he can just reject the PR and either have the AI agent refine it or start over. Or ultimately resort to writing the code himself.

He doesn't need to make the AI written code work, and so he doesn't need to spend a lot of time reading the AI written code - he can skim it for any sign it looks even faintly off and just kill it if that's the case instead of spending more time on it.

> My previous point stands - if it was that cut and dry, then a (free) script/library could generate the same code.

There's a vast chasm between simple enough that a non-AI code generator can generate it using templates and simple enough that a fast read-through is enough to show that it's okay to run.

As an example, the other day I had my own agent generate a 1kloc API client for an API. The worst case scenario other than failing to work would be that it would do something really stupid, like deleting all my files. Since it passes its tests, skimming it was enough for me to have confidence that nowhere does it do any file manipulation other than reading the files passed in. For that use, that's sufficient since it otherwise passes the tests and I'll be the only user for some time during development of the server it's a client for.

But no template based generator could write that code, even though it's fairly trivial - it involved reading the backend API implementation and rote-implementation of a client that matched the server.

Re: Generative AI coding tools and agents do not work for me

#214
post #128

I'm fine with anybody saying AI agents don't work for their work-style and am not looking to rebut this piece, but I'm going to take this opportunity to call something out. The author writes "reviewing code is actually harder than most people think. It takes me at least the same amount of time to review code not written by me than it would take me to write the code myself". That sounds within an SD of true for me, to…

This is radical and healthy way to do it. Obviously wrong — reject. Obviously right — accept. In any other case — also reject, as non-obvious. I guess it is far removed from the advertized use case. Also, I feel one would be better off having auto-complete powered by LLM in this case.

Auto-complete means having to babysit it.

The more I use this, the longer the LLM will be working before I even look at the output any more than maybe having it chug along on another screen and occasionally glance over.

My shortest runs now usually takes minutes of the LLM expanding my prompt into a plan, writing the tests, writing the code, linting its code, fixing any issues, and write a commit message before I even review things.

Re: Generative AI coding tools and agents do not work for me

#215

Earlier quoted context omitted.

It’s mostly attitude that you are learning. Playfulness, persistence and a willingness to start from scratch again and again.

>persistence and a willingness to start from scratch again and again. i.e. continually gambling and praying the model spits something out that works instead of thinking.

Most things in life are like that.

Re: Generative AI coding tools and agents do not work for me

#216
post #156

Earlier quoted context omitted.

I mostly just approve PRs because I trust my engineers. I have developed a 6th sense for thousand-line PRs and knowing which 100-300 lines need careful study. Yes I have been burned. But 99% of the time, with proper test coverage it is not an issue, and the time (money) savings have been enormous. "Ship it!" - me

I think this points out the crux of the difference of collaborating with other devs vs collaborating with am AI. The article correctly States that the AI will never learn your preferences or idiosyncrasies of the specific projects/company etc because it effectively is amnesic. You cannot trust the AI the same you trust other known collaborators because you don't have a real relationship with it.

I do something to the effect of "Update LLM.md with what you've learned" at the end of every session, coupled with telling it what is wrong when I reject a change. It works. It could work better, but it works.

Re: Generative AI coding tools and agents do not work for me

#217
I've started to finish some abandoned half-ready side projects with Claude Pro on Desktop with filesystem MCP. Used to high quality code, it took me some time to teach Claude to follow conventions. Now it works like a charm, we work on a requirements.md until all questions are answered and then I let Claude go. Only thing left is convincing clients to embrace code assistents.

Re: Generative AI coding tools and agents do not work for me

#218
post #28

Earlier quoted context omitted.

Do you have to review the code? I’ll be honest that, like the OP theorizes, I often just spot review it. But I also get it to write specs (often very good, in terms of the ones I’ve dug into), and I always carefully review and test the results. Because there is also plenty of non-AI code in my projects I didn’t review at all, namely, the myriad open source libraries I’ve installed.

It depends on what you're doing. If it's a simple task, or you're making something that won't grow into something larger, eyeballing the code and testing it is usually perfect. These types of tasks feel great with Claude Code. If you're trying to build something larger, it's not good enough. Even with careful planning and spec building, Claude Code will still paint you into a corner when it comes to architecture. In…

If you let it paint you into a corner, why are you doing so?

I don't know what you mean by "a lot of guidance". Maybe I just naturally do that, but to me there's not been much change in the level of guidance I need to give Claude Code or my own agent vs. what I'd give developers working for me.

Another issue is that as long as you ensure it builds good enough tests, the cost of telling it to just throw out the code it builds later and redo it with additional architectural guidance keeps dropping.

The code is increasingly becoming throwaway.

Post reply on HN