I read all of Cloudflare's Claude-generated commits • Hacker News Board

Re: I read all of Cloudflare's Claude-generated commits

#141

Post by maxemitchell » Sat, Jun 07, 2025, 2:50 PM UTC

fastball wrote: post #30

SrslyJosh wrote: post #5

> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…

The idea as stated is a poor one, but a slight reshuffling and it seems promising: You generate code with LLMs. You write tests for this code, either using LLMs or on your own. You of course commit your actual code: it is required to actually run the program, after all. However you also save the entire prompt chain somewhere. Then (as stated in the article), when a much better model comes along, you re-run that chain…

Your rephrasing better encompasses my idea, and I should have emphasized in the post that I do not think this is a good idea (nor possible) right now, it was more of a hand-wavy "how could we rethink source control in a post-LLM world" passing thought I had while reading through all the commits.

Clearly it struck a chord with a lot of the folks here though, and it's awesome to read the discourse.

Re: I read all of Cloudflare's Claude-generated commits

#142

Post by lcnPylGDnU4H9OF » Sat, Jun 07, 2025, 2:58 PM UTC

keybored wrote: post #80

tptacek wrote: post #44

Earlier quoted context omitted.

Most of the developers I know personally who have been radicalized by coding agents, it happened within the past 9 months. It does not feel like we are in a phase of predictable boring improvement.

Radicalized? Going with the flow and wishes of the people who are driving AI is the opposite of that. To have their minds changed drastically, sure..

> very different from the usual or traditional

https://www.merriam-webster.com/dictionary/radical

Deciding that AI is going nowhere to suddenly deciding that coding agents are how they will work going forward is a radical change. That is what they meant.

Re: I read all of Cloudflare's Claude-generated commits

#143

Post by maxemitchell » Sat, Jun 07, 2025, 2:58 PM UTC

pollinations wrote: post #108

SrslyJosh wrote: post #5

> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…

I'd say commit a comprehensive testing system with the prompts. Prompts are in a sense what higher level programming languages were to assembly. Sure there is a crucial difference which is reproducibility. I could try and write down my thoughts why I think in the long run it won't be so problematic. I could be wrong of course. I run https://pollinations.ai which servers over 4 million monthly active users quite relia…

Agreed with the medium-term solution. I wish I put some more detail into that part of the post, I have more thoughts on it but didn't want to stray too far off topic.

Re: I read all of Cloudflare's Claude-generated commits

#144

Post by maxemitchell » Sat, Jun 07, 2025, 3:03 PM UTC

7speter wrote: post #109

SrslyJosh wrote: post #5

> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…

I think the author is saying you commit the prompt with the resulting code. You said it yourself, storage is free, so comment the prompt along with the output (don’t comment that out that if I’m not being clear); it would show the developers(?) intent, and to some degree, almost always contribute to the documentation process.

Author here :). Right now, I think the pragmatic thing to do is to include all prompts used in either the PR description and/or in the commit description. This wouldn't make my longshot idea of "regenerating a repo from the ground up" possible, but it still adds very helpful context to code reviewers and can help others on your team learn prompting techniques.

Re: I read all of Cloudflare's Claude-generated commits

#145

Post by lcnPylGDnU4H9OF » Sat, Jun 07, 2025, 3:10 PM UTC

kiitos wrote: post #57

tptacek wrote: post #45

Earlier quoted context omitted.

Why do you care about hallucination for coding problems? You're in an agent loop; the compiler is ground truth. If the LLM hallucinates, the agent just iterates. You don't even see it unless you make the mistake of looking closely.

What on earth are you talking about?? If the LLM hallucinates, then the code it produces is wrong. That wrong code isn't obviously or programmatically determinable as wrong, the agent has no way to figure out that it's wrong, it's not as if the LLM produces at the same time tests that identify that hallucinated code as being wrong. The only way that this wrong code can be identified as wrong is by the human user "loo…

> You seem to have this fundamental belief that the code that's produced by your LLM is valid and doesn't need to be evaluated, line-by-line, by a human, before it can be committed??

This is a mistaken understanding. The person you responded to has written on these thoughts already and they used memorable words in response to this proposal:

> Are you a vibe coding Youtuber? Can you not read code? If so: astute point. Otherwise: what the fuck is wrong with you?

It should be obvious that one would read and verify the code before they commit it. Especially if one works on a team.

https://fly.io/blog/youre-all-nuts/

Re: I read all of Cloudflare's Claude-generated commits

#146

Post by maxemitchell » Sat, Jun 07, 2025, 3:29 PM UTC

thorum wrote: post #23

Humorous that this article has a strong AI writing smell - the author should publish the prompts they used!

I did human notes -> had Claude condense and edit -> manually edit. A few of the sentences (like the stinky one below) were from Claude which I kept if it matched my own thoughts, though most were changed for style/prose.

I'm still experimenting with it. I find it can't match style at all, and even with the manual editing it still "smells like AI" as you picked up. But, it also saves time.

My prompt was essentially "here are my old blog posts, here's my notes on reading a bunch of AI generated commits, help me condense this into a coherent article about the insights I learned"

Re: I read all of Cloudflare's Claude-generated commits

#147

Post by BurritoKing » Sat, Jun 07, 2025, 3:32 PM UTC

kevingadd wrote: post #47

tptacek wrote: post #43

Earlier quoted context omitted.

I don't know why that's funny. This is not a post about a vibe coding session. It's Kenton Varda['s coding session]. later updated to clarify kentonv didn't write this article

I think it makes sense that GP is skeptical of this article considering it contains things like: > this tool is improving itself, learning from every interaction which seem to indicate a fundamental misunderstanding of how modern LLMs work: the 'improving' happens by humans training/refining existing models offline to create new models, and the 'learning' is just filling the context window with more stuff, not enhanc…

When you consider the "tool" as more than just the LLM model, but the stuff wrapped around calling that model then I feel like you can make a good argument it's improving when it keeps context in a file on disk and constantly updates and edits that file as you work throguh the project.

I do this routinely for large initiatives I'm kicking off through Claude Code - it writes a long detailed plan into a file and as we work through the project I have it constantly updating and rewriting that document to add information we have jointly discovered from each bit of the work. That means every time I come back and fire it back up, it's got more information than when it started, which looks a lot more improvement from my perspective.

Re: I read all of Cloudflare's Claude-generated commits

#148

Post by SpicyLemonZest » Sat, Jun 07, 2025, 3:51 PM UTC

ryandrake wrote: post #38

IncreasePosts wrote: post #18

I asked this in the other thread (no response, but I was a bit late) How does anyone using AI like this have confidence that they aren't unintentionally plagiarizing code and violating the terms of whatever license it was released under? For random personal projects I don't see it mattering that much. But if a large corp is releasing code like this, one would hope they've done some due diligence that they have to jus…

This is an excellent question that the AI-boosters always seem to dance around. Three replies already are saying “Nobody cares.” Until they do. I’d be willing to bet that some time in the near future, some big company is going to care a lot and that there will be a landmark lawsuit that significantly changes the LLM landscape. Regulation or a judge is going to eventually decide the extent to which someone can use AI…

It just presumes a level of fixation in copyright law that I don’t think is realistic. There was a landmark lawsuit MAI v. Peak Computer in 1993, where judges determined that repairing a computer without the permission of the operating system’s author is copyright infringement, and it didn’t change the landscape at all because everyone immediately realized it’s not practical for things to work that way. There’s no realistic world where AI tools end up being extremely useful but nobody uses them because of a court ruling.

Re: I read all of Cloudflare's Claude-generated commits

#149

Post by QuantumNoodle » Sat, Jun 07, 2025, 4:02 PM UTC

SupremumLimit wrote: post #4

It's an interesting review but I really dislike this type of techno-utopian determinism: "When models inevitably improve..." Says who? How is it inevitable? What if they've actually reached their limits by now?

What is ironic, if we buy in to the theory that AI will write majority of the code in the next 5-10 years, what is it going to train on after? ITSELF? Seems this theoretic trajectory of "will inevitably get better" is is only true if humans are producing quality training data. The quality of code LLMs create is very well proportionate on how mature and ubiquitous the langues/projects are.

Re: I read all of Cloudflare's Claude-generated commits

#150

Post by delegate » Sat, Jun 07, 2025, 4:18 PM UTC

dcre wrote: post #60

tptacek wrote: post #54

Earlier quoted context omitted.

I don't know why people don't give more credence to the argument that the exact opposite thing will happen.

Right. I don’t understand why everyone thinks this will make it impossible for junior devs to learn. The people I had around to answer my questions when I was learning knew a whole lot less than Claude and also had full time jobs doing something other than answering my questions.

You learn by doing.. eg typing the code. It's not just knowledge, it's the intuition you develop when you write code yourself. Just like physical exercise. Or playing an instrument. It's not enough to know the theory, practice is key.

AI makes it very easy to avoid typing and hence make learning this skill less attractive.

But I don't necessarily see it as doom and gloom, what I think will happen - juniors will develop advanced intuition about using AI and getting the functionality they need, not the quality of the code, while at the same time the AI models will get increasingly better and write higher quality code.