Live data from Hacker News

Build a Basic AI Agent from Scratch: Long Task Planning

medium.com

41–50 of 60 posts

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#41

I've tried most form of planning - from the basic AGENTS.md guide to keeping ./dev/ plan files, todo list tools, sqlite db with both minimal and extensive tracking, etc. None of them have been worth it. A year ago the models needed to be reminded. Today they can follow a plan from text alone. This is my experience from working on a project alone - in teams ... i actually think the same lesson holds in the new AI para…

Please don't take offense to this very dumb question: Why can't you do the planning ? Figure out what needs to be done , break it down into small tasks and then ask the agent to execute those small tasks? When we executed projects in the past, this is what I would do as a lead: figure out the overall software architecture and delegate the tasks to developers. This way I always knew how the system worked and could ext…

People get defensive when you ask this, because the they think you’re saying they’re being lazy.

…but it’s than just that (in most cases; I am just lazy sometimes); but fundamentally there’s a limit to how much complexity people can comprehend.

We are good at working at high level abstractions, modules with clear apis that can be sprung to together into some kind of feature.

You don’t need to look inside the black box of the module if you trust the implementer; Ive never opened up the internals of a calendar be like “how does this work?”. I just don’t care. It’s a calendar. I use the api.

I think most people are using these tools in this way; very few people are having an agent write a plan, then a sub agent review it, no human in the loop. Those are for prototypes and are yolo cowboys using open claw and playing with the phones instead of working; we have a few at work, but their PRs are regularly rejected as slop.

…but, realistically; many people aren’t software architects. They may not even know coding patterns, forget architecture patterns.

Having an agent spit out generic software architecture is probably better than what they were producing before.

Writing a module / feature using generic architecture and planning is probably better than random code spaghetti right?

It’s easy to lament the loss of craft here, but at the end of the day, the models today do an ok job of this. The models of tomorrow will probably be better at it than many people.

Architecture is easy composed to actually implementing things. You just wave your hands from your ivory tower and say “more event sourcing”.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#42
What's the point of the scratch pad? Isn't the same data already in the context? Or does it help because contexts are lossy and bias towards the start and end?

Similar question with the to-do list. Do they actually help task completion? Is there any research on that? I think they're less helpful with more recent models, but maybe they still help with smaller ones?

The system prompt asking it to make a plan before starting work does sound helpful though. (Of course it would also be great to see numbers there :)

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#43
post #23

Earlier quoted context omitted.

The recovery strategy in my mind would be what to do in case of a crash, which would just wipe out all the context here (scratch pad, todo list, etc) - it doesn't seem very recoverable.

This is the difficult part of programming debates. What you mentioned is about the TODO list disappearing immediately when Python shuts down, right? What I was talking about is the point where the LLM retries when something goes wrong due to a mistake in the previous task. Actually, that's why I included the sentence 'If I were to do the code properly, I would use a separate storage instead of in memory.' I guess I u…

Yeah, I clicked through and saw that prompting but I would consider that more of a retry mechanism and wanted to clarify.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#44

I've tried most form of planning - from the basic AGENTS.md guide to keeping ./dev/ plan files, todo list tools, sqlite db with both minimal and extensive tracking, etc. None of them have been worth it. A year ago the models needed to be reminded. Today they can follow a plan from text alone. This is my experience from working on a project alone - in teams ... i actually think the same lesson holds in the new AI para…

Please don't take offense to this very dumb question: Why can't you do the planning ? Figure out what needs to be done , break it down into small tasks and then ask the agent to execute those small tasks? When we executed projects in the past, this is what I would do as a lead: figure out the overall software architecture and delegate the tasks to developers. This way I always knew how the system worked and could ext…

This is more or less what I do. Then again, I work on a small parts of the codebase at a time, so maybe the autonomous agent works better when you're doing larger refactors over large codebases.

Even in that situation, I think I would still only feel comfortable approaching the task as I would do it without AI, and using the AI to accelerate the parts that would be time-consuming. E.g. finding where/how feature X is implemented, how it would affect the overall system if I were to change it this way, etc.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#45

Earlier quoted context omitted.

Please don't take offense to this very dumb question: Why can't you do the planning ? Figure out what needs to be done , break it down into small tasks and then ask the agent to execute those small tasks? When we executed projects in the past, this is what I would do as a lead: figure out the overall software architecture and delegate the tasks to developers. This way I always knew how the system worked and could ext…

People get defensive when you ask this, because the they think you’re saying they’re being lazy. …but it’s than just that (in most cases; I am just lazy sometimes); but fundamentally there’s a limit to how much complexity people can comprehend. We are good at working at high level abstractions, modules with clear apis that can be sprung to together into some kind of feature. You don’t need to look inside the black bo…

"Having an agent spit out generic software architecture is probably better than what they were producing before."

If they were a poor programmer/architect, I don't think the AI would make the end result any better. It would amplify their lack of skill. Sure, the low-level code might be more airtight and idiomatic, but that's not even where poor skill really manifests itself. It's at the higher level of thinking in terms of the system and understanding the proper context of the business/technology, etc.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#46
post #23

Earlier quoted context omitted.

This is the difficult part of programming debates. What you mentioned is about the TODO list disappearing immediately when Python shuts down, right? What I was talking about is the point where the LLM retries when something goes wrong due to a mistake in the previous task. Actually, that's why I included the sentence 'If I were to do the code properly, I would use a separate storage instead of in memory.' I guess I u…

Yeah, I clicked through and saw that prompting but I would consider that more of a retry mechanism and wanted to clarify.

You are right. I am not being critical of you. I just wanted to say that I wrote my comment in a somewhat confusing way. English is not my native language, so it might have come across as a bit harsh

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#47
post #15
post #5

Why do people use Medium?

Because it gives them a way to post articles for free? What should they use instead, your highness? Why do people post comments like this?

What a strange comment.

The original post is also available at the poster’s own blog [1], so the question is a very valid one. Clearly, “posting articles for free” is a hurdle already cleared by the author.

[1] https://www.ruxu.dev/articles/ai/build-an-ai-agent-planning/

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#48
post #30

Earlier quoted context omitted.

Please don't take offense to this very dumb question: Why can't you do the planning ? Figure out what needs to be done , break it down into small tasks and then ask the agent to execute those small tasks? When we executed projects in the past, this is what I would do as a lead: figure out the overall software architecture and delegate the tasks to developers. This way I always knew how the system worked and could ext…

The kinds of detailed (and excellent) plans Opus or Fable can generate on our large code base would take me maybe 1-2 days to work through and they do in 10-20 minutes. Maybe I spent 2-4 hours reviewing it, checking things with colleagues etc. Then I press "go" and maybe an hour later I have a tested system ready for manual review. It's plans are at least as good as any I've seen. Their weakness is if there are unsta…

Did you manage to setup a discussion with the agent to reveal such assumptions? Sometimes the shave wrong unstated assumptions when contradicted by evidence, but if we’re taking about a plan for the future the evidence is thin.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#49
post #46

Earlier quoted context omitted.

Yeah, I clicked through and saw that prompting but I would consider that more of a retry mechanism and wanted to clarify.

You are right. I am not being critical of you. I just wanted to say that I wrote my comment in a somewhat confusing way. English is not my native language, so it might have come across as a bit harsh

No harshness detected, and yeah, even when everyone is speaking the same language the jargon is always hyper specific.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#50
post #30

Earlier quoted context omitted.

Please don't take offense to this very dumb question: Why can't you do the planning ? Figure out what needs to be done , break it down into small tasks and then ask the agent to execute those small tasks? When we executed projects in the past, this is what I would do as a lead: figure out the overall software architecture and delegate the tasks to developers. This way I always knew how the system worked and could ext…

The kinds of detailed (and excellent) plans Opus or Fable can generate on our large code base would take me maybe 1-2 days to work through and they do in 10-20 minutes. Maybe I spent 2-4 hours reviewing it, checking things with colleagues etc. Then I press "go" and maybe an hour later I have a tested system ready for manual review. It's plans are at least as good as any I've seen. Their weakness is if there are unsta…

Cognitive debt
Post reply on HN