Research-Driven Agents: When an agent reads before it codes • Hacker News Board

Re: Research-Driven Agents: When an agent reads before it codes

#21

Post by simlevesque » Thu, Apr 09, 2026, 7:41 PM UTC

paulluuk wrote: post #18

simlevesque wrote: post #11

I've been making skills from arxiv papers for a while. I have a one for multi-object tracking for example. It has a SKILL.md describing all important papers (over 30) on the subject and a folder with each paper's full content as reStructuredText. To feed Arxiv papers to LLMs I found that RST gives the best token count/fidelity ratio. Markdown lacks precision. LateX is too verbose. I have a script with the paper's url…

This sounds like it would work, but honestly if you've already read all 30 papers fully, what do you still need to llm to do for you? Just the boilerplate?

I'm trying to make a go library that implements a wide ranges of MOT algorithms and can gather metrics for all of them.

Reading all the papers once isn't the same as this. I find it very useful.

I can ask an LLM to do the basic implementations, then I can refine them (make the code better, faster, cut on memory use), then I can ask the LLM if I'm still implementing the algorithms as they're described in the paper.

Re: Research-Driven Agents: When an agent reads before it codes

#22

Post by ctoth » Thu, Apr 09, 2026, 7:41 PM UTC

I've been very interested in this recently. I'm pretty sure that every project should have a ./papers directory of annotated papers in it like I do in Qlatt[0].

Literally every project. If it's something that's been done a million times then that means it has good literature on it? If not, then even more important to find related stuff! And not just crunchy CS stuff like databases or compilers or whatever. Are you creating a UI? There's probably been great UI research you can base off of! Will this game loop be fun in the game you're building? There's probably been research about it!

[0]: https://github.com/ctoth/Qlatt/blob/master/papers/

Re: Research-Driven Agents: When an agent reads before it codes

#23

Post by [deleted] » Thu, Apr 09, 2026, 7:41 PM UTC

MrLeap wrote: post #17

simlevesque wrote: post #11

I've been making skills from arxiv papers for a while. I have a one for multi-object tracking for example. It has a SKILL.md describing all important papers (over 30) on the subject and a folder with each paper's full content as reStructuredText. To feed Arxiv papers to LLMs I found that RST gives the best token count/fidelity ratio. Markdown lacks precision. LateX is too verbose. I have a script with the paper's url…

What is RST?

[deleted]

Re: Research-Driven Agents: When an agent reads before it codes

#24

Post by [deleted] » Thu, Apr 09, 2026, 7:42 PM UTC

MrLeap wrote: post #17

simlevesque wrote: post #11

I've been making skills from arxiv papers for a while. I have a one for multi-object tracking for example. It has a SKILL.md describing all important papers (over 30) on the subject and a folder with each paper's full content as reStructuredText. To feed Arxiv papers to LLMs I found that RST gives the best token count/fidelity ratio. Markdown lacks precision. LateX is too verbose. I have a script with the paper's url…

What is RST?

[deleted]

Re: Research-Driven Agents: When an agent reads before it codes

#25

Post by ctoth » Thu, Apr 09, 2026, 7:43 PM UTC

simlevesque wrote: post #11

I've been making skills from arxiv papers for a while. I have a one for multi-object tracking for example. It has a SKILL.md describing all important papers (over 30) on the subject and a folder with each paper's full content as reStructuredText. To feed Arxiv papers to LLMs I found that RST gives the best token count/fidelity ratio. Markdown lacks precision. LateX is too verbose. I have a script with the paper's url…

I've been working on ctoth/research-papers-plugin, the pipeline to actually get LLMs to extract the notes. I really like your insight re RST over Markdown! It sounds like we're working on similar stuff and I'll absolutely reach out :)

Re: Research-Driven Agents: When an agent reads before it codes

#26

Post by alex000kim » Thu, Apr 09, 2026, 7:46 PM UTC

ctoth wrote: post #22

I've been very interested in this recently. I'm pretty sure that every project should have a ./papers directory of annotated papers in it like I do in Qlatt[0]. Literally every project. If it's something that's been done a million times then that means it has good literature on it? If not, then even more important to find related stuff! And not just crunchy CS stuff like databases or compilers or whatever. Are you cr…

That directory is huge already! I guess the index.md helps the agent find what it needs, but even the markdown file is very long - this would consume a ton of tokens.

Also I wonder who/what decides what papers go in there.

In the blog post, the agent is allowed to do its own search.

Re: Research-Driven Agents: When an agent reads before it codes

#27

Post by zzleeper » Thu, Apr 09, 2026, 7:47 PM UTC

ctoth wrote: post #22

I've been very interested in this recently. I'm pretty sure that every project should have a ./papers directory of annotated papers in it like I do in Qlatt[0]. Literally every project. If it's something that's been done a million times then that means it has good literature on it? If not, then even more important to find related stuff! And not just crunchy CS stuff like databases or compilers or whatever. Are you cr…

Wow this is amazing. Did you write all those MD files by hand, or used an LLM for the simple stuff like extracting abstracts?

Re: Research-Driven Agents: When an agent reads before it codes

#28

Post by maCDzP » Thu, Apr 09, 2026, 7:49 PM UTC

I have a ML project. I usually set up a team of agents, where I have a leader, archivist, research assistant, researcher, developer and tester. The team generates hypothesis based on papers, test it, and iterate over that. Everything is documented using a lab notebook. It burns tokens but I have found some promising strategies that I am testing.

Re: Research-Driven Agents: When an agent reads before it codes

#29

Post by simlevesque » Thu, Apr 09, 2026, 7:51 PM UTC

ctoth wrote: post #25

simlevesque wrote: post #11

I've been making skills from arxiv papers for a while. I have a one for multi-object tracking for example. It has a SKILL.md describing all important papers (over 30) on the subject and a folder with each paper's full content as reStructuredText. To feed Arxiv papers to LLMs I found that RST gives the best token count/fidelity ratio. Markdown lacks precision. LateX is too verbose. I have a script with the paper's url…

I've been working on ctoth/research-papers-plugin, the pipeline to actually get LLMs to extract the notes. I really like your insight re RST over Markdown! It sounds like we're working on similar stuff and I'll absolutely reach out :)

I'm gonna look at your plugin. My email is in my profile.

Honestly I think that Markdown with LateX code blocks would be the most efficient representation but when doing it with Pandoc I kept having issues with loss of information and sometimes even syntax error.

Re: Research-Driven Agents: When an agent reads before it codes

#30

Post by ctoth » Thu, Apr 09, 2026, 7:53 PM UTC

zzleeper wrote: post #27

ctoth wrote: post #22

I've been very interested in this recently. I'm pretty sure that every project should have a ./papers directory of annotated papers in it like I do in Qlatt[0]. Literally every project. If it's something that's been done a million times then that means it has good literature on it? If not, then even more important to find related stuff! And not just crunchy CS stuff like databases or compilers or whatever. Are you cr…

Wow this is amazing. Did you write all those MD files by hand, or used an LLM for the simple stuff like extracting abstracts?

I used https://github.com/ctoth/research-papers-plugin to produce the annotations. The thing that's really cool is how they surface the cross-links in the collection, for instance look at https://github.com/ctoth/Qlatt/blob/master/papers/Fant_1988_...

Claude is much faster and better at reading papers than Codex (some of this is nested skill dispatch) but they both work quite incredibly for this. Compile your set of papers, queue it up and hit /ingest-collection and go sleep, and come back to a remarkable knowledge base :)