Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We should be more tired than the model

vickiboykis.com

Post reply on HN
Print view View original

61–70 of 168 posts

Re: We should be more tired than the model

#61

Post by the_other »

CraigJPerry wrote: post #10

I clearly identify with the problem the author raises, which is: the bottleneck is understanding. I don't go along with their mitigations though. In programming we have one tool for this: abstraction. Decomposition, pattern recognition, even data structures and algorithms are all down stream of abstraction. Collectively, we've never truly mastered abstraction, but it's what we have and we collectively wield it well e…

The "right" abstraction seems like quite an art. Sometimes it's not obvious, or it takes multiple rounds of exploration and testing (I'm thinking here of the mental shift moving from HTML + JS, via jQuery, Backbone, Knockout and up to React/Vue or Angular). At all points, we thought we had reasonable abstractions for a while. Vue and Svelt, or NextJS, now are so far from the mental model of early 00s "DHTML".

And I'm not sure how this relates to TFA's point. Are you saying we collectively need to get better at abstraction so that LLMs get better at abstraction (either by training, or our prompting), so that their code is easier to read?

Re: We should be more tired than the model

#62

Post by askonomm »

nyrikki wrote: post #60
dpoloncsak wrote: post #51

Earlier quoted context omitted.

> Junior engineer learn, AI does not. This is technically true, but lets not act like we haven't seen immense improvement of both models are harnesses for these models in the past years. They may not be learning , but they are getting better

They are getting better at historical data, not at the fundamental issue. As a recent example, I recently had to abandon the multiple LLM reviewer/verifier model I was using because zig 0.16 was released with major changes. I actually reverted back to full self hosted because the foundation models we’re trying too hard to revert to the older versions of the language. It is going to be a balancing act and there is fun…

I find great success in not relying on LLM's built-in knowledge, but giving it links to necessary docs/manuals and have it read that before doing anything.

Re: We should be more tired than the model

#63

Post by the_other »

yondys wrote: post #17

Lately I've been thinking about this a lot. I've slightly shifted my use of Claude from implementing tool to scaffold generator for me to actually do the hard parts. It's frustrating at first, because the impulse always is "I could get Claude to do this in minutes", but that's just the brain trying to spare some energy. I've found that it's much more rewarding to use LLMs as an aid to deep work instead of a substitut…

This sounds like something I'd enjoy. Do you have a blog post or guide on your approach?

Re: We should be more tired than the model

#64

Post by simonw »

thfuran wrote: post #35
simonw wrote: post #14

Something I've been trying recently for non-throwaway code is extensive refactoring, without typing any code myself but by closely directing the coding agent. Prompts like "move the code relating to SQL query analysis into a new file", "look for opportunities to use pytest parametrize to remove duplication in that test", "rename method X to Y". Early indications are that this is helping a lot with the problem where i…

>Prompts like "move the code relating to SQL query analysis into a new file", "look for opportunities to use pytest parametrize to remove duplication in that test", "rename method X to Y". There’s a lot of overlap there with the sorts of things traditional automated refactoring tools can do approximately instantly, locally, and for free.

Sure, and sometimes the coding agent will even use one of those refactoring tools on my behalf.

Getting them to run ast-grep is really fun, especially when it saves me from having to memorize that syntax myself.

Re: We should be more tired than the model

#65

Post by simonw »

ryandrake wrote: post #59
thfuran wrote: post #35

Earlier quoted context omitted.

>Prompts like "move the code relating to SQL query analysis into a new file", "look for opportunities to use pytest parametrize to remove duplication in that test", "rename method X to Y". There’s a lot of overlap there with the sorts of things traditional automated refactoring tools can do approximately instantly, locally, and for free.

Yea, when I read about people using AI with prompts like that, my first thought is, "Wow, that's like copy/paste, but instead of Ctrl-C/Ctrl-V, it's round-tripping to a server and using GPUs to do it." What's next? "Claude, rename the function doFoo() to performBar()"?

Here's the loop for a successful small refactor (anything beyond a rename that could be handled entirely by an IDE):

1. Find the code you want to change

2. Run the tests to confirm that test coverage is good for the starting point

3. Track down everywhere else that might call or interact with that code

4. Update the tests (red/green TDD)

5. Alter the code

6. Update the things that call the code

7. Run the tests again

8. Apply linters/formatters

9. Address any feedback from linters

10. Check to see if any documentation needs updating and do that

11. Land a commit with a descriptive commit message

I can get all of that done with a coding agent with a single sentence prompt - especially if it's already in a session where it knows that I do "red/green TDD".

... and then I can work on something else while the agent is churning through those steps.

Re: We should be more tired than the model

#66

Post by simonw »

swatcoder wrote: post #55
mattacular wrote: post #40

Earlier quoted context omitted.

What are some traditional automated refactoring tools that can do stuff like those tasks from the example?

??? Mature workflows for those kinds of tasks have been mostly ubiquitous across professional-grade engineering tools like those from JetBrains or Visual Studio itself for longee than many people here have even been working in the trade. It's clearly not the case for simonw, but much of what many people task AI tools to do foe them are only a novelty for the "VS Code"-type users who stubbornly refused to explore more…

Oh I'd definitely classify myself as a "'VS Code'-type users who stubbornly refused to explore more professional-grade paid tools in the past."

I've never liked the larger IDEs - VS Code only won me over because it was indistinguishable from a lighter text editor at first, and the IDE tools then emerged slowly as I used it.

Re: We should be more tired than the model

#67

Post by xantronix »

paulmooreparks wrote: post #39

I don't know. I find that I'm moving up a level and improving my product-management skills while delegating most of the code to the agents. I'm still very much hands-on with the design and requirements, and I'm asking questions like, "What's our security story for XYZ?", "Are we accounting for colour-blindness?", etc. Not being down in the code allows me to prairie-dog a bit more and see the landscape better.

One thing I've noticed is that LLMs have allowed middle managers trapped inside the role of a developer to finally self actualise.

Re: We should be more tired than the model

#68

Post by mock-possum »

cautiouscat wrote: post #11

I agree with the article, though I will say with an agentic workflow I feel more tired at the end of it than I would doing it by hand. Maybe it’s the constant reading and digging in the generated code, or the constant context switching while waiting for it to think/generate. Or it’s both.

I’ve been noticing the same thing. I’m getting roughly 2-3x more done, but I’m also at least 1.5x more exhausted at the end of the day.

Re: We should be more tired than the model

#69

Post by embedding-shape »

askonomm wrote: post #62
nyrikki wrote: post #60

Earlier quoted context omitted.

They are getting better at historical data, not at the fundamental issue. As a recent example, I recently had to abandon the multiple LLM reviewer/verifier model I was using because zig 0.16 was released with major changes. I actually reverted back to full self hosted because the foundation models we’re trying too hard to revert to the older versions of the language. It is going to be a balancing act and there is fun…

I find great success in not relying on LLM's built-in knowledge, but giving it links to necessary docs/manuals and have it read that before doing anything.

Also, add "no assumptions or guesses" and if you use a model with really strong prompt adherence (most SOTA models), they'll figure out the right version first, then look up docs, then implement.

Re: We should be more tired than the model

#70

Post by ryandrake »

simonw wrote: post #65
ryandrake wrote: post #59

Earlier quoted context omitted.

Yea, when I read about people using AI with prompts like that, my first thought is, "Wow, that's like copy/paste, but instead of Ctrl-C/Ctrl-V, it's round-tripping to a server and using GPUs to do it." What's next? "Claude, rename the function doFoo() to performBar()"?

Here's the loop for a successful small refactor (anything beyond a rename that could be handled entirely by an IDE): 1. Find the code you want to change 2. Run the tests to confirm that test coverage is good for the starting point 3. Track down everywhere else that might call or interact with that code 4. Update the tests (red/green TDD) 5. Alter the code 6. Update the things that call the code 7. Run the tests again…

My point is that all these steps can be done very quickly by even a junior developer who knows emacs or their IDE, in a codebase with existing lint/format/test automation, without even taking their hands off the keyboard. You're already in your IDE, you can probably do it just as fast there. I don't see the cost/benefit of spending tokens and hitting a server for this kind of work.

I guess the difference may be in people's mode of AI working: Do you primarily develop in your IDE or a bunch of terminals running vim, and occasionally fire up claude to do more complex things? Or do you primarily develop in a long-lasting claude terminal, and occasionally tab over to the IDE to watch/codereview? In other words: What dev tool is on your primary monitor and what's on your secondary monitor? It's getting hard for developers in one camp to discuss coding and see eye-to-eye with developers from the other camp.

Post reply on HN