Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memorizing session transcripts isn't useful

12gramsofcarbon.com

Post reply on HN
Print view View original

101–110 of 168 posts

Re: Memorizing session transcripts isn't useful

#101

Post by lukeschlather »

At the core this is a hardware problem. 1M tokens is simply not enough context to understand a codebase the way a human would understand it. Being able to selectively forget is potentially a very valuable power, but right now it's a substitute for a human's ability to remember the rough shape of something, decide it's uninteresting, and remember that it is uninteresting.

They talk about memory only being useful when guided by a human, I think the proper solution is deeper than that, it probably involves feeding the entire codebase and every agent session into a finetuning of the model, though at that point you might want some guidance to avoid feeding certain sessions into the model. Or maybe not, maybe the bitter lesson applies.

Re: Memorizing session transcripts isn't useful

#102

Post by kelnos »

LPisGood wrote: post #16
dofm wrote: post #9

Blog posts like this just blow me away. > I believed this so strongly that my company built an entire product around this concept. I used to tell folks that "session transcripts were the new oil," that they were more valuable than the code itself. > […] > We don't really write code by hand anymore. Honestly, isn't this just influencer spam? What possible value is there in reading about people who used to have product…

I have to ask: do you still write a lot of code yourself? I and most people I know do not.

Yes, nearly all of it. Having the agent write code for me doesn't really save me much time, and the code quality is usually worse (and it takes even more time if I insist on better code quality from the agent).

And I don't think I'm unique. I see enough posts like https://news.ycombinator.com/item?id=48777257 pop up that I'm reasonably confident all the hype around LLMs saving so much time and increasing productivity so much is, well, just that: hype.

Sure, if you can't code at all and want to build something, an LLM is going to be great for you, even if you can't evaluate the code quality or determine if there are bugs just by looking at the code. But I've been coding professionally for 25 years, and as a hobby since I was like 8 years old. I like to code! It's a passion of mine. If the LLM isn't doing it faster or better (and most of the time it isn't), why wouldn't I write code myself?

I'll have the LLM write boilerplate stuff or do tedious refactoring, because I just don't feel like it (even if it does take longer). But for the real work? Of course I do most of it myself.

One area where the LLM shines for me is finding the root causes of bugs. It can generally do that much faster than I do. Often orders of magnitude faster (like minutes instead of hours or days). But when it comes to write the fix for the bug? It's usually faster and better if I do it myself.

Re: Memorizing session transcripts isn't useful

#103

Post by theahura »

charcircuit wrote: post #31

>We have found zero performance benefit on SWE tasks when agents have search access to their previous transcript sessions I refuse to believe this is true. The ability for an agent to find information from before a compaction is incredibly useful. At compaction time it's impossible to know what exactly may be still needed.

With the million-context-window models we never hit compaction, observed over hundreds of sessions. What are you doing that has you hitting compaction regularly?

Re: Memorizing session transcripts isn't useful

#104

Post by dofm »

yoyohello13 wrote: post #89
slopinthebag wrote: post #72

Earlier quoted context omitted.

It doesn’t matter what materials or techniques you use to build a house. 99.99% of construction exists to achieve business outcomes, and velocity matters a lot more than using the right materials or techniques. Of course the house must pass safety inspections and stuff, but the materials and techniques don’t matter one bit for that. All that matters is you achieve the desired outcome, and I will ignore the glaring fa…

> Of course the house must pass safety inspections and stuff, but the materials and techniques don’t matter one bit for that. All that matters is you achieve the desired outcome, and I will ignore the glaring fact that you achieve the desired outcome by using the right materials and techniques. This analogy is more true than you think. This is why modern homes/appartments are trash. You can pass safety inspections us…

It does feel like a good analogy.

e.g. the bill is definitely coming true for a lot of "non-traditional construction" materials and methods in immediately post-war properties in the UK. There are many unmortgageable properties using Mundic Block in Cornwall and to some extend Devon, in the heavily bombed south east there was a lot of pre-stressed concrete with catastrophic rebar failure, not to mention Orlit construction, and all across the country a lot of RAAC. Almost all of it for good, necessary, upbeat reasons.

It feels a bit like this kind of crisis from AI generated code could hit in ten, fifteen years time; people often fail to understand how long a bit of website code can last.

Re: Memorizing session transcripts isn't useful

#106

Post by koolba »

general_reveal wrote: post #57
dofm wrote: post #48

Earlier quoted context omitted.

Oh god, I just realised this really is the logical parallel to all those TV crime dramas set in the early 1900s.

It’ll be the programmers version of those civil war reenactments.

The two sides would be the strongly typed union and the duck typed heretics of the confederacy.

Re: Memorizing session transcripts isn't useful

#107

Post by slopinthebag »

yoyohello13 wrote: post #89
slopinthebag wrote: post #72

Earlier quoted context omitted.

It doesn’t matter what materials or techniques you use to build a house. 99.99% of construction exists to achieve business outcomes, and velocity matters a lot more than using the right materials or techniques. Of course the house must pass safety inspections and stuff, but the materials and techniques don’t matter one bit for that. All that matters is you achieve the desired outcome, and I will ignore the glaring fa…

> Of course the house must pass safety inspections and stuff, but the materials and techniques don’t matter one bit for that. All that matters is you achieve the desired outcome, and I will ignore the glaring fact that you achieve the desired outcome by using the right materials and techniques. This analogy is more true than you think. This is why modern homes/appartments are trash. You can pass safety inspections us…

Yeah I agree. And you have people on this forum who gleefully point out that quality doesn’t matter to the business, as if they think they’re so intelligent because they noticed that employees are there to make the company money. Not realizing that A) it’s a very antisocial attitude and B) it’s not a tenable long term strategy.

Re: Memorizing session transcripts isn't useful

#108

Post by csomar »

techpression wrote: post #87
csomar wrote: post #68

Earlier quoted context omitted.

I am now in the process of fixing code I wrote using AI. I have come to the realization that AI can't really write software and I am annoyed that it took me that long (months) to realize that.

This is quite terrifying to me, because I have a feeling I will soon come to the same conclusion. I’m starting to see some really glaring omissions in code I’m responsible for (using Opus) that at first (and second) look seemed fine, but really isn’t.

I talked with a friend on a different field (academic) and he had to re-review all things written by AI. Basically, he used AI to read/summarize/find stuff in large academic papers but realized later that many times AI makes glaring mistakes that on a first read pass the smell test.

Re: Memorizing session transcripts isn't useful

#109

Post by slopinthebag »

estearum wrote: post #78
slopinthebag wrote: post #72

Earlier quoted context omitted.

It doesn’t matter what materials or techniques you use to build a house. 99.99% of construction exists to achieve business outcomes, and velocity matters a lot more than using the right materials or techniques. Of course the house must pass safety inspections and stuff, but the materials and techniques don’t matter one bit for that. All that matters is you achieve the desired outcome, and I will ignore the glaring fa…

No the materials and techniques matter a lot. This is why we need to build houses with sticks and jute cord, just like we always have. It's vital also that we paint our special symbols above the door to ward off the spirits. It's insane to me that you're implying we could build houses with pre-fabricated materials or pneumatic nail guns and still somehow "have houses?" No sticks/jute cord and special symbols, then no…

The argument isn’t to not use better materials or techniques, it’s that inferior materials and techniques are fine because they don’t impact the end result, which is so obviously false when it comes to pretty much anything, but supposedly true when it comes to software.

Re: Memorizing session transcripts isn't useful

#110

Post by dofm »

jenniferhooley wrote: post #74
dofm wrote: post #21

Earlier quoted context omitted.

I am a freelancer recovering from severe burnout so the answer is a sort of irrelevant no. I'm trying to rebuild my life so I am in an experimenting and learning phase rather than a massive coding phase, and most of my code work is maintenance of things I have built. That which I do code, I am still coding by hand, though I am dealing with other people's Claude output and I am really unimpressed by it. It's often rat…

Programmers can use smaller models like deepseek v4 flash for 98% of the same productivity as SOTA models and cost (true cost) around $10-$30 a month. So I doubt most people who heavily use them are too concerned. It's only vibe/hobby coders who really need SOTA and they probably don't think about it much.

To what extent does that ameliorate the problem?

Are you not, by developing this way, making yourself more interchangeable, less indispensable, than ever before?

Post reply on HN