231–240 of 268 posts
This reminds me of the workflow I had a year ago. Miss Aider so much. Are there any good open source agents right now? Might be a good time to try one soon as Fable switches to token-based billing, which Code is designed to maximize.
Earlier quoted context omitted.
You say you can have increasingly nuanced discussions with stronger models. What I say is, when I asked Claude why he applied a certain change I didn't understand, and boy, it was a small change, he said he "reasoned from first principles" based on the code paths. But it didn't work, and when I asked, "Okay, describe the steps of your reasoning from first principles," it literally answered that it had just made it up…
You can never ask why a model did a certain thing, or what it was "thinking" when it said something - just like you can't ask a human which neurons were firing when they had a certain thought. The information just isn't available at that level. You absolutely can have deep nuanced discussions with LLMs however, you just need to better understand their strengths and weaknesses.
1. Asking a model why it did a certain thing, and
2. Expecting a human to say which neuron fired in their response.
Earlier quoted context omitted.
You say you can have increasingly nuanced discussions with stronger models. What I say is, when I asked Claude why he applied a certain change I didn't understand, and boy, it was a small change, he said he "reasoned from first principles" based on the code paths. But it didn't work, and when I asked, "Okay, describe the steps of your reasoning from first principles," it literally answered that it had just made it up…
You can never ask why a model did a certain thing, or what it was "thinking" when it said something - just like you can't ask a human which neurons were firing when they had a certain thought. The information just isn't available at that level. You absolutely can have deep nuanced discussions with LLMs however, you just need to better understand their strengths and weaknesses.
Of course you can! It might be following outdated docs or read something in legacy code and tried to follow that pattern and it'll tell you as much if you ask it in a way that actually gets you the reason instead of it thinking it needs to immediately fix the mistake.
Earlier quoted context omitted.
If you were working as a manager on a large project, how would you build a model? Something where your position requires you to have an overview of the project but not necessarily to actually write or review much code.
I am not able to find it now, but there was an amazing story recently from the 60s or 70s where an engineer was in exactly this position. His team was building a new, complex, ambitious operating system, but it was late and over budget and didn’t work. It nearly wrecked the company. He talks about hitting rock bottom and asking himself what went wrong, and one of the fathers of computing (can’t remember which) shouts…
I don't think we can do both. The difference is that it's optional now depending on the project and the audience.
Earlier quoted context omitted.
I am not able to find it now, but there was an amazing story recently from the 60s or 70s where an engineer was in exactly this position. His team was building a new, complex, ambitious operating system, but it was late and over budget and didn’t work. It nearly wrecked the company. He talks about hitting rock bottom and asking himself what went wrong, and one of the fathers of computing (can’t remember which) shouts…
I never said you wouldn't have to read code. I was asking a question to get answers about how people would achieve having an oversight if it was humans writing code that they were managing rather than agents. As for your suggestion, understanding every line might have worked in the 70s but even pre-agentic modern coding it's not possible for any large project with dependencies even if you are directly contributing co…
I thought it was going to be even shorter leash - code autocomplete with smaller local models. That raises the level of interactivity and leads to better code knowledge.
Certainly, if you want to be even more involved in writing the code, more power to ya!
Earlier quoted context omitted.
Unfortunately I ran into the same issue even before AI. The forgetting curve means that my mental model lasts not much longer than the initial building period. As for how to rebuild it, I haven't figured that part out yet.
Sounds like you wrote very poor quality (edit: or trivial) code, or you’re exaggerating a bit for effect. I too forget the details of most of the code I write, but the most important 10-20% of the code that I write encodes my mental model of the problem I’m trying to solve. Sometimes it’s a class representation of a digital or physical entity. Sometimes it’s a job with tasks that map to subproblems. Those abstraction…
I understand each piece and what it talks to. But I can't hold them all in my mind at once, because there's too many pieces. (I think chunking helps here, but it seems to require a certain level of fluency with the entire codebase that I'm not sure it's feasible to hit with anything past a certain line count. I am working on this new memory software though...)
The transformer on the other hand, just loads it into context (they can do about 10K LoC these days without performance degradation), cross references everything against everything (that's how the transformer works! That's why they're so expensive) and just tells me what talks to what, what the full chain is, and also btw you have 3 bugs you didn't notice because they involve how distant parts of the chains interact, you're welcome!
I've been looking for ways to build up that mental model. The Feynman technique seemed like a good place to start. I did it on a section of my codebase. It took half an hour of poking around to connect all the pieces. The transformer was able to do it instantly.
I'm not sure if there was added value to me poking around manually or if those 30 minutes would have been better spent just memorizing what it told me.
(After verification of course! To clarify, I don't think they're infallible, but their perception is broader than ours due to how they're structured, and I'm learning to utilize that more effectively.)
Also, in the absence of that costly verification, the model my Feynman technique produced turned out to be wrong (though it sounded correct!). So I'm leaning in the direction of, the way to actually verify your mental model is to make a modification to the codebase. Make reality push back!
--
On some projects/subprojects I do build an explicit mental model beforehand, and then I do generally remember it pretty well, at least for a while. Others take a more iterative approach to the design. (I'm on the 5th damn iteration of my netcode right now.)
So there's two distinct issues here, the model building process and the human forgetting curve.