Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Research-Driven Agents: When an agent reads before it codes

blog.skypilot.co

Post reply on HN
Print view View original

61–70 of 81 posts

Re: Research-Driven Agents: When an agent reads before it codes

#61

Post by technotony »

dalmo3 wrote: post #60
jbergqvist wrote: post #41

When I want to solve a new problem with an agent, I always ask it to search broadly for prior work in the given area online, and then analyze if we can build our solution using it as inspiration. I see it as the solution being out there in “idea space”, and by having the agent search beforehand we can more efficiently explore this space before converging on the final solution.

Is it not safe to assume that all* publicly available prior work is in the training data? Then you could just prompt it to propose options with pros and cons etc. * Bar extremely new stuff from after the cutoff

Maybe but having it search first to load the context with relevant information sure gets better results

Re: Research-Driven Agents: When an agent reads before it codes

#62

Post by giancarlostoro »

simlevesque wrote: post #21
paulluuk wrote: post #18

Earlier quoted context omitted.

This sounds like it would work, but honestly if you've already read all 30 papers fully, what do you still need to llm to do for you? Just the boilerplate?

I'm trying to make a go library that implements a wide ranges of MOT algorithms and can gather metrics for all of them. Reading all the papers once isn't the same as this. I find it very useful. I can ask an LLM to do the basic implementations, then I can refine them (make the code better, faster, cut on memory use), then I can ask the LLM if I'm still implementing the algorithms as they're described in the paper.

> then I can ask the LLM if I'm still implementing the algorithms as they're described in the paper.

Unit testing would save on tokens... unit testing is perfect for validating refactors, or when re-writing a project from one language to the next, build unit tests first.

Re: Research-Driven Agents: When an agent reads before it codes

#64

Post by timClicks »

ctoth wrote: post #25
simlevesque wrote: post #11

I've been making skills from arxiv papers for a while. I have a one for multi-object tracking for example. It has a SKILL.md describing all important papers (over 30) on the subject and a folder with each paper's full content as reStructuredText. To feed Arxiv papers to LLMs I found that RST gives the best token count/fidelity ratio. Markdown lacks precision. LateX is too verbose. I have a script with the paper's url…

I've been working on ctoth/research-papers-plugin, the pipeline to actually get LLMs to extract the notes. I really like your insight re RST over Markdown! It sounds like we're working on similar stuff and I'll absolutely reach out :)

Another format that's worth investigating is Asciidoc. It supports the richness of Docbook XML but has fewer quirks than rST in my eyes.

Re: Research-Driven Agents: When an agent reads before it codes

#67

Post by SadErn »

dalmo3 wrote: post #60
jbergqvist wrote: post #41

When I want to solve a new problem with an agent, I always ask it to search broadly for prior work in the given area online, and then analyze if we can build our solution using it as inspiration. I see it as the solution being out there in “idea space”, and by having the agent search beforehand we can more efficiently explore this space before converging on the final solution.

Is it not safe to assume that all* publicly available prior work is in the training data? Then you could just prompt it to propose options with pros and cons etc. * Bar extremely new stuff from after the cutoff

[dead]
Post reply on HN