Live data from Hacker News

Generative AI coding tools and agents do not work for me

blog.miguelgrinberg.com

141–150 of 464 posts

Re: Generative AI coding tools and agents do not work for me

#141

Earlier quoted context omitted.

How are those without that kind of scratch supposed to keep up with those that do?

What makes you think those without that kind of scratch are supposed to keep up?

For the past 10 years we have been telling everyone learn to code, now it’s learn to build AI prompts.

Before a poor kid with a computer access could learn to code nearly for free, but if it costs $1k just to get started with AI that poor kid will never have that opportunity.

Re: Generative AI coding tools and agents do not work for me

#142
post #10

Earlier quoted context omitted.

You can pre-agree on a solution path with human engineers too, with a similar effect.

Don't try to argue with those using AI coding tools. They don't interact well with actual humans, which is why they've been relegated to talking to the computer. We'll eventually have them all working on some busy projects to help with "marketing" to keep them distracted while the decent programmers that can actually work in a team environment can get back to useful work free of the terrible programmers and marketing…

> that can actually work in a team environment can get back to useful work free of the terrible programmers

Is that what you and your buddies talk about at two hour long coffee/smoke breaks while “terrible” programmers work?

Re: Generative AI coding tools and agents do not work for me

#143

Earlier quoted context omitted.

I feel like there is also a very high ceiling to how much scaffolding you can produce for the agents to get them to work better. This includes custom prompts, custom CLAUDE.md files, other documentation files for Claude to read, and especially how well and quickly your linting and tests can run, and how much functionality they cover. That's not to mention MCP and getting Claude to talk to your database or open your w…

When you say specification, what, specifically, does that mean? Do you have an example? I've actually been playing around with languages that separate implementation from specification under the theory that it will be better for this sort of stuff, but that leaves an extremely limited number of options (C, C++, Ada... not sure what else). I've been using C and the various LLMs I've tried seem to have issues with the…

Like a spec you'd hand to a contractor. List of requirements, some business context, etc. Not a formal algorithm spec.

My basic initial prompt for that is: "we're creating a markdown specification for (...). I'll start with basic description and at each step you should refine the spec to include the new information and note what information is missing or could use refinement."

Re: Generative AI coding tools and agents do not work for me

#144

You could argue that AI-generated code is a black box, but let's adjust our perspective here. When was the last time you thoroughly reviewed the source code of a library you imported? We already work with black boxes daily as we evaluate libraries by their interfaces and behaviors, not by reading every line. The distinction isn't whether code comes from AI or humans, but how we integrate and take responsibility for i…

> When was the last time you thoroughly reviewed the source code of a library you imported?

Doesn't matter, I'm not responsible for maintaining that particular code

The code in my PRs has my name attached, and I'm not trusting any LLM with my name

Re: Generative AI coding tools and agents do not work for me

#145

"do not work for me", I believe, is the key message here. I think a lot of AI companies have crafted their tools such that adoption has increased as the tools and the output got better. But there will always be a few stragglers, non-normative types, or situations where the AI agent is just not suitable.

Maybe, but there's also some evidence that AI coding tools aren't making anyone more productive. One study from last year found that there was no increase in developer velocity but a dramatic increase in bugs.[1] Granted, the technology has advanced since this study, but many of the fundamental issues of LLM unreliability remain. Additionally, a recent study has highlighted the significant cognitive costs associated…

One year ago I probably would've said the same. But I started dabbling with it recently, and I'm awed by it.

Re: Generative AI coding tools and agents do not work for me

#146

> It takes me at least the same amount of time to review code not written by me than it would take me to write the code myself, if not more. Hard disagree. It's still way faster to review code than to manually write it. Also the speed at which agents can find files and the right places to add/edit stuff alone is a game changer.

There's a difference between reviewing code by developers you trust, and reviewing code by developers you don't trust or AI you don't trust. Although tbh, even in the worse case I think I am still faster at reviewing than writing. The only difference is though, those reviews will never have had the same depth of thought and consideration as when I write the code myself. So reviews are quicker, but also less thorough/…

> also less thorough/robust than writing for me.

This strikes me as a tradeoff I'm absolutely not willing to make, not when my name is on the PR

Re: Generative AI coding tools and agents do not work for me

#147

Earlier quoted context omitted.

I feel like there is also a very high ceiling to how much scaffolding you can produce for the agents to get them to work better. This includes custom prompts, custom CLAUDE.md files, other documentation files for Claude to read, and especially how well and quickly your linting and tests can run, and how much functionality they cover. That's not to mention MCP and getting Claude to talk to your database or open your w…

When you say specification, what, specifically, does that mean? Do you have an example? I've actually been playing around with languages that separate implementation from specification under the theory that it will be better for this sort of stuff, but that leaves an extremely limited number of options (C, C++, Ada... not sure what else). I've been using C and the various LLMs I've tried seem to have issues with the…

A "specification" as in a text document outlining all the changes to make.

For example, it might include: Overview, Database Design (Migration, Schema Updates), Backend Implementation (Model Updates, API updates), Frontend Implementation (Page Updates, Component Design), Implementation Order, Testing Considerations, Security Considerations, Performance Considerations.

It sounds like a lot when I type it out, but it is pretty quick to read through and edit.

The specification document is generated by a planning prompt that tells Claude to analyse the feature description (the couple paragraphs I wrote), research the repository context, research best practices, present a plan, gather specific requirements, perform quality control, and finally generate the planning document.

I'm not sure if this is the best process, but it seems to work pretty well.

Re: Generative AI coding tools and agents do not work for me

#148
post #79

Earlier quoted context omitted.

There's an implied assumption here that developers who end up spending all their time reviewing LLM code won't lose their skills or become homicidal. :p

Fair enough. ;-) I'm actually curious about the "lose their skills" angle though. In the open source community it's well understood that if anything reviewing a lot of code tends to sharpen your skills.

I expect that comes from the contrast and synthesis between how the author is anticipating things will develop or be explained, versus what the other person actually provided and trying to understand their thought process.

What happens if the reader no longer has enough of that authorial instinct, their own (opinionated) independent understanding?

I think the average experience would drift away from "I thought X was the obvious way but now I see by doing Y you were avoid that other problem, cool" and towards "I don't see the LLM doing anything too unusual compared to when I ask it for things, LGTM."

Re: Generative AI coding tools and agents do not work for me

#149
post #90

Earlier quoted context omitted.

You do not need to review every line of every package you use, just the subset of the interface you import/link and use. You have to review every line of code you commit into your project. I think attempting to equate the two is dishonest dissembling.

To me, the point the friend is making is, just like you said, that you don't need to review every line of code in a package, just the interface. The author misses the point that there truly is code that you trust without seeing it. At the moment AI code isn't as trustworthy as a well tested package but that isn't intrinsic to the technology, just a byproduct of the current state. As AI code becomes more reliable, it…

This absolutely is intrinsic to the workflow

Using a package that hundreds of thousands of other people use is low risk, it is battle tested

It doesn't matter how good AI code gets, a unique solution that no one else has ever touched is always going to be more brittle and risky than an open source package with tons of deployments

And yes, if you are using an Open Source package that has low usage, you should be reviewing it very carefully before you embrace it

Treat AI code as if you were importing from a git repo with 5 installs, not a huge package with Mozilla funding

Re: Generative AI coding tools and agents do not work for me

#150
post #45

Earlier quoted context omitted.

I do not agree it is something you can pick up in an hour. You have to learn what AI is good at, how different models code, how to prompt to get the results you want. If anything, prompting well is akin to learning a new programming language. What words do you use to explain what you want to achieve? How do you reference files/sections so you don't waste context on meaningless things? I've been using AI tools to code…

I am also interested in how much of these skills are at the mercy of OpenAI ? Like IIRC 1 or 2 years ago there was an uproar of AI "artists" saying that their art is ruined because of model changes ( or maybe the system prompt changed ). >I do not agree it is something you can pick up in an hour. But it's also interesting that the industry is selling the opposite ( with AI anyone can code / write / draw / make music…

Of course that's what the industry is selling because they want to make money. Yes, it's easy to create a proof of concept but once you get out of greenfield into 50-100k tokens needed in the context (reading multiple 500 line files, thinking, etc) the quality drops and you need to know how to focus the models to maintain the quality.

"Write me a server in Go" only gets you so far. What is the auth strategy, what endpoints do you need, do you need to integrate with a library or API, are there any security issues, how easy is the code to extend, how do you get it to follow existing patterns?

I find I need to think AND write more than I would if I was doing it myself because the feedback loop is longer. Like the article says, you have to review the code instead of having implicit knowledge of what was written.

That being said, it is faster for some tasks, like writing tests (if you have good examples) and doing basic scaffolding. It needs quite a bit of hand holding which is why I believe those with more experience get more value from AI code because they have a better bullshit meter.

Post reply on HN