Live data from Hacker News

We put a coding agent in a while loop

github.com

131–140 of 317 posts

Re: We put a coding agent in a while loop

#131

Earlier quoted context omitted.

It hints that a suitable auto completion of the input prompt is to output a pkill command

We, too, are just auto-complete, next-token machines.

We are auto-complete next-token machines, but plastic and attached to many other not less important subsystems, which is a crucial difference.

Re: We put a coding agent in a while loop

#132

There are always two major results from any software development process: a change in the code and a change in cognition for the people who wrote the code (whether they did so directly or with an LLM). Python and Typescript are elaborate formal languages that emerged from a lengthy process of development involving thousands of people around the world over many years. They are non-trivially different, and it's neat th…

It's so much easier to build a mental model of a code base with LLMs. You just ask specific questions of a subsystem and they show files, code snippets, point out the idea, etc.

I just recently took the time to understood how the GIL works exactly in CPython, because I just asked a couple of questions about it, Claude showed me the relevant API and examples where can I find it. I looked it up in the CPython codebase and all of a sudden it clicked.

The huge difference was that it cost me MINUTES. I didn't even bother to dig in before, because I can't perfectly read C, the CPython codebase is huge and it would have taken me a really long time to understand everything.

Re: We put a coding agent in a while loop

#133

There are always two major results from any software development process: a change in the code and a change in cognition for the people who wrote the code (whether they did so directly or with an LLM). Python and Typescript are elaborate formal languages that emerged from a lengthy process of development involving thousands of people around the world over many years. They are non-trivially different, and it's neat th…

> But my hunch is that most of the economic value of code is contingent on there being a set of human beings familiar with the code in a manner that requires writing having written it directly. This reminds me of a software engineering axiom: When making software, remember that it is a snapshot of your understanding of the problem. It states to all, including your future-self, your approach, clarity, and appropriaten…

Yes! But there's code and code. Not to disrespect anyone, but there is writing a new algorithm, say for optimizing the gradient descent and code to display a simple web form.

The first one is usually short and requires a very deep understanding of one or two profound, new ideas. The second is usually very big and requires a shallow understanding of many not-so-new ideas (which are usually a reflection of the oroganisation that produced the code).

My feeling is that, provided a sufficiently long context window, an LLM will be able to go through the second kind project very easily. It will also be very good at showing that the first kind of project is not so new after all, destroying all people who can't find really new ideas.

In both case, it'll pressure institutions to have less IT specialists...

As someone who trained specifically in computer sciences, I'm a bit scared :-/

Re: We put a coding agent in a while loop

#134

Earlier quoted context omitted.

Superfund repos.

What makes you so sure it will have a repo? I don’t recall the last time Claude suggested anything about version control :-)

Claude will give what you asked for. My sensible chuckle moment was when I asked it to create a demo asp net web API and it did everything but add the authorize tag or any kind of authentication. I asked what was missing and until i mentioned it, it didn't mention authentication or authorization at all.

Re: We put a coding agent in a while loop

#135

Earlier quoted context omitted.

Would love to hear more about your work and how you have tapped into that market if you're keen to share. Even if it's just anecdotes about vibe-in-production gone wrong, that would be really entertaining.

Absolutely. Before vibe coding became too much of a thing we had the majority of our business coming from poorly developed web applications coming from off shore shops. That’s been more or less the last decade. Once LLMs became popular we started to see more business on that front which you would expect. What we didn’t expect is that we started seeing MUCH more “deep” work wherein the threat actor will get into core…

> We also see a ton of re-compromised environments. Why? They don’t know how to use CICD and just recommit the vulnerable code.

This one sticks out to me. A while back the UK did a security assessment of Huawei with a view to them being a core infrastructure provider for the 5G rollout, and the conclusion wasn't that they were insecure, it was that they were ~10 years away from being able to even claim they were secure.

Contrasting this to my current employer, where the software supply chain and provenance is exceptional, it's clear to me that vibe coding doesn't get you far in terms of that supply chain, and is arguably a significant regression from the norm.

Third party dependencies, runtime environments/containers, build processes, build environments, dev machines, source control, configuration, binaries, artifact signing and provenance, IDEs, none of these have good answers in the vibe-coded ecosystem and many are harmed by it. It will be interesting to see how the industry grapples with this when someone eventually pushes back and says they won't use your software because you don't have enough context about it to even claim it's secure.

Re: We put a coding agent in a while loop

#136
post #81

Earlier quoted context omitted.

I wonder though. One of the superpowers of LLMs is code reading. I say the tools are better and reading than writing. It is very easy to get comprehensive documentation for any code base and get understanding by asking questions. At that point does it matter that there is a living developer who understands the code? If an arbitrary person with knowledge of the technology stack can get up to speed quickly is it import…

i spend a lot of time thinking about this. At humanlayer we have some OSS projects that are 99% written by AI, and a lot of it was written by AI under the supervision of developer(s) that are no longer at the company. Every now and then we find that there are gaps in our own understanding of the code/architecture that require getting out the old LSP and spelunking through call stacks. It's pretty rare though.

> It's pretty rare though.

It will only get more common with time.

Re: We put a coding agent in a while loop

#137

Earlier quoted context omitted.

Superfund repos.

What makes you so sure it will have a repo? I don’t recall the last time Claude suggested anything about version control :-)

"As per my last email that contained the code claude wrote in a .pdf file I would like you to ask to fix two different users being able to see each others data if they are logged in at the same time, thank you for your attention in this matter."

Re: We put a coding agent in a while loop

#138

Earlier quoted context omitted.

Would love to hear more about your work and how you have tapped into that market if you're keen to share. Even if it's just anecdotes about vibe-in-production gone wrong, that would be really entertaining.

Absolutely. Before vibe coding became too much of a thing we had the majority of our business coming from poorly developed web applications coming from off shore shops. That’s been more or less the last decade. Once LLMs became popular we started to see more business on that front which you would expect. What we didn’t expect is that we started seeing MUCH more “deep” work wherein the threat actor will get into core…

[deleted]

Re: We put a coding agent in a while loop

#139

There will be a a new kind of job for software engineers, sort of like a cross between working with legacy code and toxic site cleanup. Like back in the day being brought in to “just fix” a amalgam of FoxPro-, Excel-, and Access-based ERP that “mostly works” and only “occasionally corrupts all our data” that ambitious sales people put together over last 5 years. But worse - because “ambitious sales people” will no lo…

I think we're already there [0].

[0] https://x.com/PovilasKorop/status/1959590015018652141

Im really curious about what other jobs will pop up. As long as there is an element of probability associated with AI, there will need to be manual supervision for certain tasks/jobs.

Re: We put a coding agent in a while loop

#140

"At one point we tried “improving” the prompt with Claude’s help. It ballooned to 1,500 words. The agent immediately got slower and dumber. We went back to 103 words and it was back on track." Isn't this the exact opposite of every other piece of advice we have gotten in a year? Another general feedback just recently, someone said we need to generate 10 times, because one out of those will be "worth reviewing" How ca…

Its not the exact opposite of what ive been reading. Basically every person claiming to have success with LLM coding that ive read have said that too long of a prompt leads to too much context which leads to the LLM diverging from working on the problem as desired.
Post reply on HN