Ask HN: Anyone struggling to get value out of coding LLMs?

I too did this (although on a small project), and I was incredibly impressed. My problem with it is that I first did it myself, and it was fairly quick and easy. The hard part was figuring out that there was a bug, and how exactly the bug behaved. The LLM helped with the easy part, but I don't know how to even explain the difficult part to it. There was no way to know which repo the problem is in, or that it wasn't a user error.

Working with AI centric IDE on the mature codebase needs different skill set that I suspect related to people management. As the pattern would be describing a problem well, making a good plan, delegate and giving feedback.

On the other side, getting a good flow is not trivial. I had to tweak rules, how to describe problem, how to plan the work and how to ask the agent. It takes time to be productive.

Eg. Asking agent to create a script to do string manipulation is better than asking them to do inplace edit. As it's easier to debug and repeat.

  It's easy to get good productivity out of LLMs in complex apps, here are my tips:

  Create a directory in the root of your project called /specs

  Chat with a LLM to drill into ideas having it play the role of a Startup Advisor, work through problem definitions, what your approach is, and build a high level plan.

  If you are happy with the state of your direction, ask the LLM to build a product-strategy.md file with the sole purpose of describing to an AI Agent what the goal of the project is.

  Discuss with an LLM all sorts of issues like:

    Components of the site

    Mono Repo vs targeted repos  

    Security issues and your approach to them

    High level map of technologies you will use

    Strong references to KISS rules and don't over complicate

    A key rule is do not build features for future use


  Wrap that up in a spec md file

  Continue this process until you have a detailed spec, with smaller .md files indexed from your main README.md spec file

  Continue promoting the role of AI Developer, AI Consumer, AI Advisor, End User

  Break all work into Phase 1 (MVP), Phase 2, and future phases, don't get more granular (only do Phase 2 if needed)

  Ask LLM to document best practice development standards and document in your CLAUDE.md or whatever you use. Discuss the standards, err to industry standard if you are lost

  Challenge the LLM while building standards, keep looping back and adjusting earlier assumptions

  Instruct AI Agents like Claude Code to target on specific spec files and implement only Phase 1. If you get stuck on how to do that, ask an LLM on how to prompt your coding agent to focus, you will learn how they operate.

  Always ask the coding agent to review any markdown files used to document your solution and update with current features, progress, and next issues.

  Paste all .md files back into other AI's e.g. high models of ChatGPT and ask it to review and identify missing areas / issues

  Don't believe everything the agents say, challenge them and refuse to let them make you happy or confirm your actions, that is not their job.

  Always provide context around errors that you want to solve, read the error, read the line number, paste in the whole function or focus your Cursor.ai prompt to that file.

  Work with all the AI's, each has their strength.

  Don't use the free models, pay, it's like running a business with borrowed tools, don't.

  Learn like crazy, there are so many tips I'm nowhere near learning.

  Be kind to your agent

(Edited: formatting)

If I could offer another suggestion from what's been discussed so far - try Claude Code - they are doing something different than the other offerings around how they manage context with the LLM and the results are quite different than everything else.

Also, the big difference with this tool is that you spend more time planning, don't expect it to 1 shot, you need to think about how you go from epic to task first, THEN you let it execute.

> This is a great example of why using LLMs to learn is perilous. This is nonsense.

These type of answers from teachers, co-students, web communities, blogs etc. are – I would assume – why people ask LLMs in the first place.