Live data from Hacker News

Memorizing session transcripts isn't useful

12gramsofcarbon.com

31–40 of 168 posts

Re: Memorizing session transcripts isn't useful

#31
>We have found zero performance benefit on SWE tasks when agents have search access to their previous transcript sessions

I refuse to believe this is true. The ability for an agent to find information from before a compaction is incredibly useful. At compaction time it's impossible to know what exactly may be still needed.

Re: Memorizing session transcripts isn't useful

#32
I like the memory system, in general. For reference I'm using mostly Opus 4.8 + Max effort. It will often pull things out of memory that are relevant. Like I'll ask it to come up with a few options I should consider for, say, a self-hosted OIDC provider and it'll say things like "Considering the size of your operations team, this might be a better fit because of X and Y".

Now, I'll agree that this is probably the sort of thing I should put in the CLAUDE.md, but in this case it wasn't on my radar to put that in my CLAUDE.md, so it was nice that it surfaced that.

It does sometimes go awry though. Today I was asking about a problem I was having authenticating, and it said "you may be running into this trusted proxy setting because you put your apps behind an haproxy". That is true of 95% of our apps, so it was worth mentioning, but in this case it was not so I had to correct it. But, I'm glad it mentioned it because if we did have it proxied it could have saved me a lot of time.

Re: Memorizing session transcripts isn't useful

#33
post #21

Earlier quoted context omitted.

I have to ask: do you still write a lot of code yourself? I and most people I know do not.

I am a freelancer recovering from severe burnout so the answer is a sort of irrelevant no. I'm trying to rebuild my life so I am in an experimenting and learning phase rather than a massive coding phase, and most of my code work is maintenance of things I have built. That which I do code, I am still coding by hand, though I am dealing with other people's Claude output and I am really unimpressed by it. It's often rat…

> That which I do code, I am still coding by hand, though I am dealing with other people's Claude output and I am really unimpressed by it. It's often rather crass.

Unfortunately the point of code is rarely to impress people (certainly not other engineers) or to avoid being "crass." 99.99% of code exists to achieve business outcomes, and velocity matters a lot in many contexts. A lot more than elegance or impressiveness.

The platform risk is a valid concern but alleviated by China's theft and redistribution of open models.

Re: Memorizing session transcripts isn't useful

#34
post #28
post #21

Earlier quoted context omitted.

I am a freelancer recovering from severe burnout so the answer is a sort of irrelevant no. I'm trying to rebuild my life so I am in an experimenting and learning phase rather than a massive coding phase, and most of my code work is maintenance of things I have built. That which I do code, I am still coding by hand, though I am dealing with other people's Claude output and I am really unimpressed by it. It's often rat…

Worst case scenario you just switch to a free model, which are 2025-ish in quality.

The open weights models I am interested in, and testing, learning, experimenting with etc.; I am confused and cynical, not insane.

I am not convinced it isn't vulnerable to the same problems but the whole tenor of the community around open source/open weights models just doesn't have the same YOLO madness to it.

Re: Memorizing session transcripts isn't useful

#35
post #21

Earlier quoted context omitted.

I have to ask: do you still write a lot of code yourself? I and most people I know do not.

I am a freelancer recovering from severe burnout so the answer is a sort of irrelevant no. I'm trying to rebuild my life so I am in an experimenting and learning phase rather than a massive coding phase, and most of my code work is maintenance of things I have built. That which I do code, I am still coding by hand, though I am dealing with other people's Claude output and I am really unimpressed by it. It's often rat…

Personally I use 5 different model families, 3 of which are open weights with 3rd party inference providers (GLM, DeepSeek, Kimi), so if the frontier labs were to shut down it'd be a nuisance, nothing more.

Re: Memorizing session transcripts isn't useful

#36
post #9

Blog posts like this just blow me away. > I believed this so strongly that my company built an entire product around this concept. I used to tell folks that "session transcripts were the new oil," that they were more valuable than the code itself. > […] > We don't really write code by hand anymore. Honestly, isn't this just influencer spam? What possible value is there in reading about people who used to have product…

Look man, I’ve got a MMO that I’m working on that’s set in 2014 where everyone is a programmer in SV (might call it World of Legacy). It’s a period piece. I NEED as much blog training data of this type so that my NPCs can talk in a historically accurate way (god bless Medium.com, a historical treasure trove of a bygone medieval era).

It’s gonna be a living breathing world, you see. You’re going to be like “omg, this game even accurately captured the blog posts, woah”.

Edit:

This whole blogging-about-cloud-AI genre is just weird and irresponsible now)

I sincerely never considered it was a whole genre.

Re: Memorizing session transcripts isn't useful

#37

Isn’t this just a form of the bitter lesson? Our attempts to make engineered context and agents will simply be made obsolete with bigger and better models. Those transcripts are probably extremely useful for lesser capable models, and near unnecessary for frontier ones, maybe?

Yeah, the question is whether this applies to all of context management.

I've been using a custom harness based on https://minimal-agent.com/ (itself based on swe-mini-agent), which is like 50 lines for the core logic. Bash is all you need.

For small tasks, I find it's about 8x faster (and uses 8x fewer tokens) than the standard harness for each model.

For bigger tasks I haven't tested it much. It seems to work too but I think they're a bit less focused and productive in that case. It could be that those big harnesses' 20k token system prompts are doing something important with regard to steering software development workflows. (e.g. I heard Fable has a custom system prompt in Claude Code which might explain its markedly more proactive behavior.)

So I want to say there's still a lot of value in context engineering though it seems to diminish with each model release (since they're fine tuned on mostly non stupid behavior and need less hand holding).

Re: Memorizing session transcripts isn't useful

#38
post #9

Blog posts like this just blow me away. > I believed this so strongly that my company built an entire product around this concept. I used to tell folks that "session transcripts were the new oil," that they were more valuable than the code itself. > […] > We don't really write code by hand anymore. Honestly, isn't this just influencer spam? What possible value is there in reading about people who used to have product…

I have to ask: do you still write a lot of code yourself? I and most people I know do not.

I am. I have Codex running, doing some tasks which I don't care much about, but anything I want to understand I write myself.

Same thing with hobby projects - I might ask ChatGPT or Gemini some questions about best practices in Swift for example, but writing code is done by hand.

As others said - if you don't use it, you'll lose it. And I'd rather keep my skills up to date.

Re: Memorizing session transcripts isn't useful

#39
post #8

I agree with the take not to bother with a sophisticated memory system. Anything worth remembering should be in docs, guides, source comments, commit messages or tickets. You don't need another layer, every conceivable granularity is already covered by existing best practices

There is some value to agents being able to query the history of work done, docs aren't a good place to accumulate negative evidence for example, but it can be tagged in traces so that it's efficient to look up as needed. Additionally, docs rot while traces can be tagged with commit hashes and other things that make their lifetime clearer.

Re: Memorizing session transcripts isn't useful

#40
post #21

Earlier quoted context omitted.

I am a freelancer recovering from severe burnout so the answer is a sort of irrelevant no. I'm trying to rebuild my life so I am in an experimenting and learning phase rather than a massive coding phase, and most of my code work is maintenance of things I have built. That which I do code, I am still coding by hand, though I am dealing with other people's Claude output and I am really unimpressed by it. It's often rat…

> That which I do code, I am still coding by hand, though I am dealing with other people's Claude output and I am really unimpressed by it. It's often rather crass. Unfortunately the point of code is rarely to impress people (certainly not other engineers) or to avoid being "crass." 99.99% of code exists to achieve business outcomes, and velocity matters a lot in many contexts. A lot more than elegance or impressiven…

I'm not talking about impressing people.

We used to be concerned about code quality. Are we not anymore?

Crassness was a signal. Still is, to me — in a human I find that people who write crass code are going to cause me trouble.

Post reply on HN