Live data from Hacker News

Build a Basic AI Agent from Scratch: Long Task Planning

medium.com

51–60 of 60 posts

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#51
post #47
post #15

Earlier quoted context omitted.

Because it gives them a way to post articles for free? What should they use instead, your highness? Why do people post comments like this?

What a strange comment. The original post is also available at the poster’s own blog [1], so the question is a very valid one. Clearly, “posting articles for free” is a hurdle already cleared by the author. [1] https://www.ruxu.dev/articles/ai/build-an-ai-agent-planning/

Someone else explained it better, if you genuinely don't understand:

https://news.ycombinator.com/item?id=48489337

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#52
post #31

Earlier quoted context omitted.

how many datacenters / computers are there running millions of hours of computer games? Why is escaping reality and damaging climate with compute better than using LLMs?

Because it's more fun? Being serious this is a silly line of reasoning. Maybe they are both bad? It's like asking why is it bad to light a forest fire when there is a forest fire already burning. I take issue with the cognitive dissonance too though. HN became very hostile to Bitcoin but took no issue with people gaming on PCs and consoles that were consuming more and more electricity each year. Now everyone is silen…

Exactly, people like to cherry pick. Do we need facebook, do we need instagram? Do we need on-demand always available 4k streaming?

The singularity argument makes more sense than the environmental one.

Short of all of us living like the amish, we're hypocrites for pointing out one "waste" over another.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#53
post #42

What's the point of the scratch pad? Isn't the same data already in the context? Or does it help because contexts are lossy and bias towards the start and end? Similar question with the to-do list. Do they actually help task completion? Is there any research on that? I think they're less helpful with more recent models, but maybe they still help with smaller ones? The system prompt asking it to make a plan before sta…

Hi, There are few benefits from using scratchpad or any other external platform : - Be agnostic of the LLM you use, tomorow, if the prices of the llm you use are exploding, you can still reuse another LLM by pointing it the scratchpad repository you have. Then, modulo the level of verbosity you had on scratchpad (or other), you'll avoid lossing time ro reexplain everything to the new llm - You can avoid the "summarized" effect obtained through context compaction events . This effect makes accurate and so potential important information a bit more lurry (numbers turned into adjectives, etc/ Scratchpad or Obsidian or any other external solution you might imagine would act as "case fact blocks" that are a solution recommended to mitigate the above effect and thus make the accurate information still available. You can imagine a system where you ask your LLM to read some files within your external storage after each compaction for exemple with a hook or anything else.

Regarding the todolist, from my pov, it's just a basic principle of work segmentation and accuracy with some traceability. You are better when you can divide your work into chunks that can be followed individually rather than with a huge block of work. That can also be used within the "ralph wiggum" loop pattern that might help the llm to get a goal and thus iterate until goal completion. There are few articles explaining the concept if that interests you

Hope it helps a bit !

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#54

Earlier quoted context omitted.

People get defensive when you ask this, because the they think you’re saying they’re being lazy. …but it’s than just that (in most cases; I am just lazy sometimes); but fundamentally there’s a limit to how much complexity people can comprehend. We are good at working at high level abstractions, modules with clear apis that can be sprung to together into some kind of feature. You don’t need to look inside the black bo…

"Having an agent spit out generic software architecture is probably better than what they were producing before." If they were a poor programmer/architect, I don't think the AI would make the end result any better. It would amplify their lack of skill. Sure, the low-level code might be more airtight and idiomatic, but that's not even where poor skill really manifests itself. It's at the higher level of thinking in te…

This simply isn't true anymore.

High level generic advice from agents is often, in my experience significantly better, unmodified, than doing nothing.

Obviously its better to do it properly, but you know… opus 4.8 is a pretty great model.

You might be surprised at the quality of the planning, architecture and task breakdown that a simple prompt with some context hints can give you.

…at the end of the day, if I’m working with someone and they give me 6/10 plans based on AI instead of stupid/10 plans they dreamed up, or 0/10 plans they didn't even bother (or in too much of a hurry) to write; Ill take it.

Tragedy of the commons? /shrug

You gotta be pragmatic. It turns subpar contributors into useful contributors.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#55
post #52

Earlier quoted context omitted.

Because it's more fun? Being serious this is a silly line of reasoning. Maybe they are both bad? It's like asking why is it bad to light a forest fire when there is a forest fire already burning. I take issue with the cognitive dissonance too though. HN became very hostile to Bitcoin but took no issue with people gaming on PCs and consoles that were consuming more and more electricity each year. Now everyone is silen…

Exactly, people like to cherry pick. Do we need facebook, do we need instagram? Do we need on-demand always available 4k streaming? The singularity argument makes more sense than the environmental one. Short of all of us living like the amish, we're hypocrites for pointing out one "waste" over another.

Yeah I don’t know if I come to the same conclusion as you.

There is no probabilistic way to determine if the singularity will be a positive or negative for humanity. Climate change is a net negative for humanity.

Regardless it’s clear what course we are on.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#56
post #48
post #30

Earlier quoted context omitted.

The kinds of detailed (and excellent) plans Opus or Fable can generate on our large code base would take me maybe 1-2 days to work through and they do in 10-20 minutes. Maybe I spent 2-4 hours reviewing it, checking things with colleagues etc. Then I press "go" and maybe an hour later I have a tested system ready for manual review. It's plans are at least as good as any I've seen. Their weakness is if there are unsta…

Did you manage to setup a discussion with the agent to reveal such assumptions? Sometimes the shave wrong unstated assumptions when contradicted by evidence, but if we’re taking about a plan for the future the evidence is thin.

> Did you manage to setup a discussion with the agent to reveal such assumptions?

This is what the plan review is for.

Usually it will have something like "modify abc.ts to update the widget number in the wnx collection" and I'm "hang on - why does that need to be updated when XYZ" and that subsequent discussion will reveal assumptions that are not shared.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#57
post #30

Earlier quoted context omitted.

The kinds of detailed (and excellent) plans Opus or Fable can generate on our large code base would take me maybe 1-2 days to work through and they do in 10-20 minutes. Maybe I spent 2-4 hours reviewing it, checking things with colleagues etc. Then I press "go" and maybe an hour later I have a tested system ready for manual review. It's plans are at least as good as any I've seen. Their weakness is if there are unsta…

Cognitive debt

Non sequitur

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#58
post #33

I've tried most form of planning - from the basic AGENTS.md guide to keeping ./dev/ plan files, todo list tools, sqlite db with both minimal and extensive tracking, etc. None of them have been worth it. A year ago the models needed to be reminded. Today they can follow a plan from text alone. This is my experience from working on a project alone - in teams ... i actually think the same lesson holds in the new AI para…

Did you read the article? It's not about enhancing Claude. This article is about creating your own agent, and giving it the ability to create plans and tasks list for its or. The way Claude code creates plans and tasks list for itself. The article is about creating that in your own harness for things not using claude code, like say a custom LLM integration in your own web app.

I'm not talking about enhancing claude either. The article opens saying they already implemented read/write for files. My comment is saying putting tools for planning in an agent's context is less useful than you might think.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#59

I've tried most form of planning - from the basic AGENTS.md guide to keeping ./dev/ plan files, todo list tools, sqlite db with both minimal and extensive tracking, etc. None of them have been worth it. A year ago the models needed to be reminded. Today they can follow a plan from text alone. This is my experience from working on a project alone - in teams ... i actually think the same lesson holds in the new AI para…

Please don't take offense to this very dumb question: Why can't you do the planning ? Figure out what needs to be done , break it down into small tasks and then ask the agent to execute those small tasks? When we executed projects in the past, this is what I would do as a lead: figure out the overall software architecture and delegate the tasks to developers. This way I always knew how the system worked and could ext…

I think we'd be talking past each other in terms of what "planning" means, but i wrote this anyways:

You're wrong about what i mean with delegating architecture to coding agent.

I'll let the coding agent take the first shot at it, already having in my mind a decent idea about how i'd do that. Worst case its wrong i need to correct it, more than half the time it comes up with the same sort of design, sometimes it comes up with a better alternative.

Additionally, the same pattern of: "sometimes wrong, mostly good, sometimes better" also plays out wrt naming things. I thought i was decent at naming things, but an the LLM is literally build on turning 'concepts' in a vector space into words.

And in a very real way the names its choosing will 'compress' the ideas so that the next time an LLM reads it is more likely to understand.

For this to work though you need your complete system accessible and well structured.

You say "I always knew how the system worked and could extend it as needed". If an AI can't learn how your system works then that's a problem with the system setup, not the AI. An AI can find its way in the linux kernel or chromium source code just fine.

If you're in a role where you only spend time planning / architecture, then i assume things are pretty gnarly to begin with. The thing i can only guess at - and which is on a spectrum - how much of our role exists to support the weight of accidental vs essential complexity.

i.e. can the engineers not do the planning because: they're not that good, or its very broad things that need to expertly interplay with each other, or because the org has a mountain of buried bodies.

In my experience some of the more fanatic AI people are blind to the mountain of buried bodies covering a lot of essential complexity, but others can be blind to how well AI works when you can just shoot of a prompt to unbury a body and actually reduce the debt.

But in one sentence:

> Why can't you do the planning ?

This way lets me do more planning - planning is basically all i do now.

Re: Build a Basic AI Agent from Scratch: Long Task Planning

#60
post #51
post #47

Earlier quoted context omitted.

What a strange comment. The original post is also available at the poster’s own blog [1], so the question is a very valid one. Clearly, “posting articles for free” is a hurdle already cleared by the author. [1] https://www.ruxu.dev/articles/ai/build-an-ai-agent-planning/

Someone else explained it better, if you genuinely don't understand: https://news.ycombinator.com/item?id=48489337

That is in fact a better explanation due to bringing up different reasons (zero cost to host as you mentioned, vs. network/visibility out-of-the-box in the linked comment).
Post reply on HN