Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I program with agents

crawshaw.io

Post reply on HN
Print view View original

11–20 of 308 posts

Re: How I program with agents

#11

Post by closewith »

quantumHazer wrote: post #8
closewith wrote: post #3

Earlier quoted context omitted.

It seems like an excellent name, given that people understand it so readily, but what else would you suggest? LoopGPT?

I’m no better at naming things! Shall we propose LLM feedback loop systems? It’s more grounded in reality. Agent is like Retina Display to my ears, at least at this stage!

Agent is clear in that it acts on behalf of the user.

"LLM feedback loop systems" could be to do with training, customer service, etc.

> Agent is like Retina Display to my ears, at least at this stage!

Retina is a great name. People know what it means - high quality screens.

Re: How I program with agents

#12

Post by layer8 »

closewith wrote: post #3
quantumHazer wrote: post #2

Finally some serious writing about LLMs that doesn’t follow the hype and it faces reality of what can and can’t be useful with these tools. Really interesting read, although I can’t stand the word “agent” for a for-loop that call recursively an LLM, but this industry is not famous for being sharp with naming things, so here we are. edit: grammar

It seems like an excellent name, given that people understand it so readily, but what else would you suggest? LoopGPT?

RePT

Re: How I program with agents

#13

Post by vmg12 »

zOneLetter wrote: post #7

Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's…

Here are the cases where it helps me (I promise this isn't ai generated even though im using a list...)

- Formulaic code. It basically obviates the need for macros / code gen. The downside is that they are slower and you can't just update the macro and re-generate. The upside is it works for code that is slightly formulaic but has some slight differences across implementations that make macros impossible to use.

- Using apis I am familiar with but don't have memorized. It saves me the effort of doing the google search and scouring the docs. I use typed languages so if it hallucinates the type checker will catch it and I'll need to manually test and set up automated tests anyway so there are plenty of steps where I can catch it if it's doing something really wrong.

- Planning: I think this is actually a very under rated part of llms. If I need to make changes across 10+ files, it really helps to have the llm go through all the files and plan out the changes I'll need to make in a markdown doc. Sometimes the plan is good enough that with a few small tweaks I can tell the llm to just do it but even when it gets some things wrong it's useful for me to follow it partially while tweaking what it got wrong.

Edit: Also, one thing I really like about llm generated code is that it maintains the style / naming conventions of the code in the project. When I'm tired I often stop caring about that kind of thing.

Re: How I program with agents

#14

Post by qsort »

voidUpdate wrote: post #4

I wonder how many people that use agents actually like "programming", as in coming up with a solution to the problem and then being able to express that in code. It seems like a lot of the work that the agents are doing is removing that and instead making you have to explain what you want in natural language and hope the LLM doesn't introduce bugs

I have to flip the question, what is it that people like about it? I certainly don't enjoy writing code for problems that have already been solved a thousand times. We reach for a dictionary, we don't write a hash table from scratch every time, that's only fun the first time you do it.

If I could go "give me a working compiler for this language" or "solve this problem using a depth-first search" I wouldn't enjoy programming any less.

About the natural language and also in response to the sibling comment, I agree, natural language is a very poor tool to describe computational processes. It's like doing math in plain English, fine for toy examples, but at a certain level of sophistication it's way too easy to say imprecise or even completely contradictory things. But nobody here advocates using LLMs "blind"! You're still responsible for your own output, whether it was generated or not.

Re: How I program with agents

#15

Post by divan »

zOneLetter wrote: post #7

Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's…

On one codebase I work with, there are often tasks that involve changing multiple files in a relatively predictable way. Like there is little creativity/challenge, but a lot of typing in multiple parts/files. Tasks like these used to take 3-4 hours complete before just because I had to physically open all these files, find right places to modify, type the code etc. With AI agent I just describe the task, and it does the job 99% correct, reducing the time from 3-4 hours to 3-4 minutes.

Re: How I program with agents

#16

Post by minikomi »

quantumHazer wrote: post #8
closewith wrote: post #3

Earlier quoted context omitted.

It seems like an excellent name, given that people understand it so readily, but what else would you suggest? LoopGPT?

I’m no better at naming things! Shall we propose LLM feedback loop systems? It’s more grounded in reality. Agent is like Retina Display to my ears, at least at this stage!

A downward spiral

Re: How I program with agents

#17

Post by bubblyworld »

hombre_fatal wrote: post #10
voidUpdate wrote: post #4

I wonder how many people that use agents actually like "programming", as in coming up with a solution to the problem and then being able to express that in code. It seems like a lot of the work that the agents are doing is removing that and instead making you have to explain what you want in natural language and hope the LLM doesn't introduce bugs

I like writing code, and it definitely isn't satisfying when an LLM can one-shot a parser that I would have had fun building for hours. But at the same time, building a parser for hours is also a distraction from my higher level ambitions with the project, and I get to focus on those. I still get to stub out the types and function signatures I want, but the LLM can fill them in and I move on. More likely I'll even ha…

I agree, agents have really made programming fun for me again (and I say this as someone who has been coding for more two decades - I'm not a script kiddy using them to make up for lack of skill).

Configuring tools, mindless refactors, boilerplate, basic unit/property testing, all that routine stuff is a thing of the past for me now. It used to be a serious blocker for me with my personal projects! Getting bored before I got anywhere interesting. Much of the time I can stick to writing the fun/critical code now and glue everything else together with LLMs, which is awesome.

Some people obviously like the fiddly stuff though, and more power to them, it's just not for me.

Re: How I program with agents

#18

Post by monkeydust »

bArray wrote: post #9

LLMs for code review, rather than code writing/design could be the killer feature. I think that code review has been broken for a while now, but this could be a way forward. Of particular interest would be security, undefined behaviour, basic misuse of features, double checking warnings out of the compiler against the source code to ensure it isn't something more serious, etc. My current use of LLMs is typically via…

Why isn't this spoken more about? Not a developer but work very closely with many - they are all on a spectrum from zero interest in this technology to actively using it to write code (correlates inversely seniority from my sample set) - very little talk on using it for reviews/checks - perhaps that needs to be done passively on commit.

Re: How I program with agents

#19

Post by rectang »

bArray wrote: post #9

LLMs for code review, rather than code writing/design could be the killer feature. I think that code review has been broken for a while now, but this could be a way forward. Of particular interest would be security, undefined behaviour, basic misuse of features, double checking warnings out of the compiler against the source code to ensure it isn't something more serious, etc. My current use of LLMs is typically via…

ChatGPT is great for debugging common issues that have been written about extensively on the web (before the training cutoff). It's a synthesizer of Stack Overflow and greatly cuts down on the time it takes to figure out what's going on compared with searching for discussions and reading them individually.

(This IP rightly belongs to the Stack Overflow contributors and is licensed to Stack Overflow. It ought to be those parties who are exploiting it. I have mixed feelings about participating as a user.)

However, the LLM output is also noisy because of hallucinations — just less noisy than web searching.

I imagine that an LLM could assess a codebase and find common mistakes, problematic function/API invocations, etc. However, there would also be a lot of false positives. Are people using LLMs that way?

Re: How I program with agents

#20

Post by Verdex »

hombre_fatal wrote: post #10
voidUpdate wrote: post #4

I wonder how many people that use agents actually like "programming", as in coming up with a solution to the problem and then being able to express that in code. It seems like a lot of the work that the agents are doing is removing that and instead making you have to explain what you want in natural language and hope the LLM doesn't introduce bugs

I like writing code, and it definitely isn't satisfying when an LLM can one-shot a parser that I would have had fun building for hours. But at the same time, building a parser for hours is also a distraction from my higher level ambitions with the project, and I get to focus on those. I still get to stub out the types and function signatures I want, but the LLM can fill them in and I move on. More likely I'll even ha…

Parsing is an area that I'm interested in. Can you talk more about your experience getting LLMs to one-shot parsers?

From scratch LLMs seem to be completely lost writing parsers. The bleeding edge appears to be able to maybe parse xml, but gives up on programming languages with even the most minimal complexity (an example being C where Gemini refused to even try with macros and then when told to parse C without macros gave an answer with several stubs where I was supposed to fill in the details).

With parsing libraries they seem better, but ultimately that reduces to transform this bnf. Which if I had to I could do deterministically without an LLM.

Also, my best 'successes' have been along the lines of 'parse in this well defined language that just happens to have dozens if not hundreds of verbatim examples on github'. Anytime I try to give examples of a hypothetical language then they return a bunch of regex that would not work in general.

Post reply on HN