Earlier quoted context omitted.
It hints that a suitable auto completion of the input prompt is to output a pkill command
We, too, are just auto-complete, next-token machines.
We put a coding agent in a while loop
131–140 of 317 posts
Re: We put a coding agent in a while loop
#132There are always two major results from any software development process: a change in the code and a change in cognition for the people who wrote the code (whether they did so directly or with an LLM). Python and Typescript are elaborate formal languages that emerged from a lengthy process of development involving thousands of people around the world over many years. They are non-trivially different, and it's neat th…
I just recently took the time to understood how the GIL works exactly in CPython, because I just asked a couple of questions about it, Claude showed me the relevant API and examples where can I find it. I looked it up in the CPython codebase and all of a sudden it clicked.
The huge difference was that it cost me MINUTES. I didn't even bother to dig in before, because I can't perfectly read C, the CPython codebase is huge and it would have taken me a really long time to understand everything.
Re: We put a coding agent in a while loop
#133There are always two major results from any software development process: a change in the code and a change in cognition for the people who wrote the code (whether they did so directly or with an LLM). Python and Typescript are elaborate formal languages that emerged from a lengthy process of development involving thousands of people around the world over many years. They are non-trivially different, and it's neat th…
> But my hunch is that most of the economic value of code is contingent on there being a set of human beings familiar with the code in a manner that requires writing having written it directly. This reminds me of a software engineering axiom: When making software, remember that it is a snapshot of your understanding of the problem. It states to all, including your future-self, your approach, clarity, and appropriaten…
The first one is usually short and requires a very deep understanding of one or two profound, new ideas. The second is usually very big and requires a shallow understanding of many not-so-new ideas (which are usually a reflection of the oroganisation that produced the code).
My feeling is that, provided a sufficiently long context window, an LLM will be able to go through the second kind project very easily. It will also be very good at showing that the first kind of project is not so new after all, destroying all people who can't find really new ideas.
In both case, it'll pressure institutions to have less IT specialists...
As someone who trained specifically in computer sciences, I'm a bit scared :-/
Re: We put a coding agent in a while loop
#134Earlier quoted context omitted.
Superfund repos.
What makes you so sure it will have a repo? I don’t recall the last time Claude suggested anything about version control :-)
Re: We put a coding agent in a while loop
#135Earlier quoted context omitted.
Would love to hear more about your work and how you have tapped into that market if you're keen to share. Even if it's just anecdotes about vibe-in-production gone wrong, that would be really entertaining.
Absolutely. Before vibe coding became too much of a thing we had the majority of our business coming from poorly developed web applications coming from off shore shops. That’s been more or less the last decade. Once LLMs became popular we started to see more business on that front which you would expect. What we didn’t expect is that we started seeing MUCH more “deep” work wherein the threat actor will get into core…
This one sticks out to me. A while back the UK did a security assessment of Huawei with a view to them being a core infrastructure provider for the 5G rollout, and the conclusion wasn't that they were insecure, it was that they were ~10 years away from being able to even claim they were secure.
Contrasting this to my current employer, where the software supply chain and provenance is exceptional, it's clear to me that vibe coding doesn't get you far in terms of that supply chain, and is arguably a significant regression from the norm.
Third party dependencies, runtime environments/containers, build processes, build environments, dev machines, source control, configuration, binaries, artifact signing and provenance, IDEs, none of these have good answers in the vibe-coded ecosystem and many are harmed by it. It will be interesting to see how the industry grapples with this when someone eventually pushes back and says they won't use your software because you don't have enough context about it to even claim it's secure.
Re: We put a coding agent in a while loop
#136Earlier quoted context omitted.
I wonder though. One of the superpowers of LLMs is code reading. I say the tools are better and reading than writing. It is very easy to get comprehensive documentation for any code base and get understanding by asking questions. At that point does it matter that there is a living developer who understands the code? If an arbitrary person with knowledge of the technology stack can get up to speed quickly is it import…
i spend a lot of time thinking about this. At humanlayer we have some OSS projects that are 99% written by AI, and a lot of it was written by AI under the supervision of developer(s) that are no longer at the company. Every now and then we find that there are gaps in our own understanding of the code/architecture that require getting out the old LSP and spelunking through call stacks. It's pretty rare though.
It will only get more common with time.
Re: We put a coding agent in a while loop
#137Earlier quoted context omitted.
Superfund repos.
What makes you so sure it will have a repo? I don’t recall the last time Claude suggested anything about version control :-)
Re: We put a coding agent in a while loop
#138Earlier quoted context omitted.
Would love to hear more about your work and how you have tapped into that market if you're keen to share. Even if it's just anecdotes about vibe-in-production gone wrong, that would be really entertaining.
Absolutely. Before vibe coding became too much of a thing we had the majority of our business coming from poorly developed web applications coming from off shore shops. That’s been more or less the last decade. Once LLMs became popular we started to see more business on that front which you would expect. What we didn’t expect is that we started seeing MUCH more “deep” work wherein the threat actor will get into core…
Re: We put a coding agent in a while loop
#139There will be a a new kind of job for software engineers, sort of like a cross between working with legacy code and toxic site cleanup. Like back in the day being brought in to “just fix” a amalgam of FoxPro-, Excel-, and Access-based ERP that “mostly works” and only “occasionally corrupts all our data” that ambitious sales people put together over last 5 years. But worse - because “ambitious sales people” will no lo…
[0] https://x.com/PovilasKorop/status/1959590015018652141
Im really curious about what other jobs will pop up. As long as there is an element of probability associated with AI, there will need to be manual supervision for certain tasks/jobs.
Re: We put a coding agent in a while loop
#140"At one point we tried “improving” the prompt with Claude’s help. It ballooned to 1,500 words. The agent immediately got slower and dumber. We went back to 103 words and it was back on track." Isn't this the exact opposite of every other piece of advice we have gotten in a year? Another general feedback just recently, someone said we need to generate 10 times, because one out of those will be "worth reviewing" How ca…