Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I program with agents

crawshaw.io

Post reply on HN
Print view View original

111–120 of 308 posts

Re: How I program with agents

#111

Post by dontlikeyoueith »

mlinhares wrote: post #22
vmg12 wrote: post #13

Earlier quoted context omitted.

Here are the cases where it helps me (I promise this isn't ai generated even though im using a list...) - Formulaic code. It basically obviates the need for macros / code gen. The downside is that they are slower and you can't just update the macro and re-generate. The upside is it works for code that is slightly formulaic but has some slight differences across implementations that make macros impossible to use. - Us…

The downside for formulaic code kinda makes the whole thing useless from my perspective, I can't imagining a case where that works. Maybe a good case, that i've used a lot, is using "spreadsheet inputs" and teaching the LLM to produce test cases/code based on the spreadsheet data (that I received from elsewhere). The data doesn't change and the tests won't change either so the LLM definitely helps, but this isn't cod…

> Maybe a good case, that i've used a lot, is using "spreadsheet inputs" and teaching the LLM to produce test cases/code based on the spreadsheet data (that I received from elsewhere)

This seems weird to me instead of just including the spreadsheet as a test fixture.

Re: How I program with agents

#112

Post by furyofantares »

I have put a lot of effort into learning how to program with agents. There was some up-front investment before the payoff. I think I'm still learning a lot, but I'm also well over the hump, the payoff has been wonderful.

The first thing I did, some months ago now, was tried to vibe code an ~entire game. I picked the smallest game design I did that I would still consider a "full game". I started probably 6 or 7 times, experimenting with different frameworks/game engines to use to find what would be good for an LLM, experimenting with different initial prompts, and different technical guidance, all in service of making something the LLM is better at developing against. Once I got settled on a good starting point and good framework, I managed to get it across the finish line with only a little bit of reading the code to get the thing un-stuck a few times.

I definitely got it done much faster and noticeably worse than if I had done it all manually. And I ended up not-at-all an expert in the system that was produced. There were times when I fought the LLM which I know was not optimal. But the experiment was to find the limits doing as little coding myself as possible, and I think (at the time) I found them.

So at that point, I've experienced three different modes of programming. Bespoke mode, which I've been doing for decades. Chat mode, where you do a lot of bespoke mode but sometimes talk to ChatGPT and paste stuff back and forth. And then nearly full vibe mode.

And it was very clear that none of these is optimal, you really want to be more engaged than vibe mode. My current project is an experiment in figuring this part out. You want to prevent the system from spiraling with bad code, and you want to end up an expert in the system that's produced. Or at least that's where I am for now. And it turns out, for me, to be quite difficult to figure out how to get out of vibe mode without going all the way to chat mode. Just a little bit of vibing at the wrong time can really spiral the codebase and give you a LOT of work to understand and fix.

I guess the impression I want to leave here is this stuff is really powerful, but you should probably expect that, if you want to get a lot of benefit out of it, there's a learning curve. Some of my vibe coding has been exhilarating, and some has been very painful, but the payoff has been huge.

Re: How I program with agents

#113

Post by ofjcihen »

Joof wrote: post #85
svaha1728 wrote: post #5

I completely agree with the author's comment that code review is half-hearted and mostly broken. With agents, the bottleneck is really in reading code, not writing it. If everyone is just half-heartedly reviewing code, or using it as a soapbox for their individual preferences, using agents will completely fall apart as they can easily introduce serious security issues or performance hits. Let's be honest, many of tho…

Isn't that the point of agents? Assume we have excellent test coverage -- the AI can write the code and ensure get the feedback for it being secure / fast / etc. And the AI can help us write the damn tests!

No, it can’t. Partially stems from the garbage the models were trained on.

Example anecdata but since we started having our devs heavily use agents we’ve had a resurgence of mostly dead vulnerabilities such as RCEs (CVE in 2019 for example) as well as a plethora of injection issues.

When asked how these made it in devs are responding with “I asked the LLM and it said it was secure. I even typed MAKE IT SECURE!”

If you don’t sufficiently understand something enough then you don’t know enough to call bs. In cases like this it doesn’t matter how many times the agent iterates.

Re: How I program with agents

#114

Post by mgraczyk »

zOneLetter wrote: post #7

Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's…

When you write code, you have to spend time on ALL of the code, no matter how simple or obvious it is.

When you read code, you can allocate your time to the parts that are more complex or important.

Re: How I program with agents

#115

Post by ofjcihen »

silverlake wrote: post #108
ofjcihen wrote: post #105

Earlier quoted context omitted.

“ It did in 10 minutes what would take me several days to learn and cobble together a bad solution.” Another way to look at this is you’re outsourcing your understanding to something that ultimately doesn’t think. This means 2 things: your solution could be severely suboptimal in multiple areas such as security and two because you didn’t bother understanding it yourself you’ll never be able to identify that. You migh…

No one is an expert on all the things. I use libraries and tools to take care of things that are less important. I use my brain for things that are important. LLMs are another tool, more flexible and capable than any other. So yes, grandpa goes to Best Buy because he’s running his legal practice and doesn’t need to be an expert on computers.

True, but I bet grandpa knows enough to identify when a paralegal has made a case losing mistake ;)

Re: How I program with agents

#116

Post by dataviz1000 »

collingreen wrote: post #90
dataviz1000 wrote: post #59

Earlier quoted context omitted.

I am beginning to love working like this. Plan a design for code. Explain to the LLM the steps to arrive to a solution. Work on reading, understanding, fixing, planing, ect. while the LLM is working on the next section of code. We are working in parallel. Think of it like being a cook in a restaurant. The order comes in. The cook plans the steps to complete the task of preparing all the elements for a dish. The cook…

I like your take and the metaphors are good at helping demonstrate by example. One caveat I wonder about is how this kind of constant context switching combines with the need to think deeply (and defensively with non humans). My gut says I'd struggle at also being the brain at the end of the day instead of just the director/conductor. I've actively paired with multiple people at once before because of a time crunch (…

> I've actively paired with multiple people at once

That was my favorite part of being a professional cook, working closely on a team.

Humans are social animals who haven't -- including how our brains are wired -- changed much physiologically in the past 25,000 years. Smart people today are not much smarter than smart people in Greece 3,000 years ago, except for the sample size of 8B people being larger. We are wired to work in groups like hunters taking down a wooly mammoth.[0]

[0] https://sc.edu/uofsc/images/feature_story_images/2023/featur...

Re: How I program with agents

#117

Post by worldsayshi »

buffalobuffalo wrote: post #70
zOneLetter wrote: post #7

Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's…

I kinda consider it a P!=nP type thing. If I need to write a simple function, it will almost always take me more time to implement it than it will to verify if an implementation of it suits my needs. There are exceptions, but overall when coding with LLMs this seems to hold true. Asking the LLM to write the function then checking it's work is a time saver.

I think this perspective is kinda key. Shifting attention towards more and better ways to verify code can probably lead to improved quality instead of degraded.

Re: How I program with agents

#118

Post by worldsayshi »

marvstazar wrote: post #103
zOneLetter wrote: post #7

Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's…

As a senior developer you already spend a significant amount of time planning new feature implementations and reviewing other people's code (PRs). I find that this skill transitions quite nicely to working with coding agents.

Exactly!

Re: How I program with agents

#119

Post by bob1029 »

zOneLetter wrote: post #7

Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's…

My most productive use of LLMs has been to stub out individual methods and have them fill in the implementations. I use a prompt like:

  public T MyMethod(/*args*/) /*type constraints*/
  {
    //TODO: Implement this method using the following requirements:
    //1 ...
    //2 ...
    //...
  }
Anything beyond this and I can't keep track of which rabbit is doing what anymore.

Re: How I program with agents

#120

Post by valcron1000 »

silverlake wrote: post #96
zOneLetter wrote: post #7

Maybe it's because I only code for my own tools, but I still don't understand the benefit of relying on someone/something else to write your code and then reading it, understand it, fixing it, etc. Although asking an LLM to extract and find the thing I'm looking for in an API Doc is super useful and time saving. To me, it's not even about how good these LLMs get in the future. I just don't like reading other people's…

You’re clinging to an old model of work. Today an LLM converted my docker compose infrastructure to Kubernetes, using operators and helm charts as needed. It did in 10 minutes what would take me several days to learn and cobble together a bad solution. I review every small update and correct it when needed. It is so much more productive. I’m driving a tractor while you are pulling an ox cart.

> It did in 10 minutes what would take me several days to learn

> I review every small update and correct it when needed

How can you review something that you don't know? How do you know this is the right/correct result beyond "it looks like it works"?

Post reply on HN