I read all of Cloudflare's Claude-generated commits • Hacker News Board

Re: I read all of Cloudflare's Claude-generated commits

#131

Post by torben-friis » Sat, Jun 07, 2025, 1:40 PM UTC

SrslyJosh wrote: post #5

> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…

Plus, commits depend on the current state of the system.

What sense does “getting rid of vulnerabilities by phasing out {dependency}” make, if the next generation of the code might not rely on the mentioned library at all? What does “improve performance of {method}” mean if the next generation used a fully different implementation?

It makes no sense whatsoever except for a vibecoders script that’s being extrapolated into a codebase.

Re: I read all of Cloudflare's Claude-generated commits

#132

Post by tptacek » Sat, Jun 07, 2025, 1:46 PM UTC

danielbln wrote: post #130

tptacek wrote: post #129

Earlier quoted context omitted.

That's not what "senior" means.

Maybe you could enlighten the rest of us then. According to your favorite definition, what does senior mean, what does seniority mean, and what's a term for someone who knows what they're doing?

Seniority means you've held the role for a long time.

Re: I read all of Cloudflare's Claude-generated commits

#133

Post by Arainach » Sat, Jun 07, 2025, 1:51 PM UTC

>Around the 40-commit mark, manual commits became frequent

This matches my experience: some shiny (even sometimes impressive) greenspace demos but dramatically less useful maintaining a codebase - which for any successful product is 90% of the work.

Re: I read all of Cloudflare's Claude-generated commits

#134

Post by etothet » Sat, Jun 07, 2025, 1:55 PM UTC

tptacek wrote: post #129

danielbln wrote: post #128

Earlier quoted context omitted.

Have a better term for "knows what they're doing" other than senior?

That's not what "senior" means.

This is not necessarily true in practical terms when it comes to hiring or promoting. Often a senior dev becomes a senior because of having an advanced skillset, despite years on the job. Similarily, often developers who have been on the job for many years aren’t ready for senior because of their lack or soft and hard skills.

Re: I read all of Cloudflare's Claude-generated commits

#135

Post by tptacek » Sat, Jun 07, 2025, 2:00 PM UTC

etothet wrote: post #134

tptacek wrote: post #129

Earlier quoted context omitted.

That's not what "senior" means.

This is not necessarily true in practical terms when it comes to hiring or promoting. Often a senior dev becomes a senior because of having an advanced skillset, despite years on the job. Similarily, often developers who have been on the job for many years aren’t ready for senior because of their lack or soft and hard skills.

Oh, that's one of the ways a senior dev becomes senior.

Re: I read all of Cloudflare's Claude-generated commits

#136

Post by skydhash » Sat, Jun 07, 2025, 2:27 PM UTC

tptacek wrote: post #123

kiitos wrote: post #104

Earlier quoted context omitted.

Irrelevant, really. Tests establish a minimum threshold of acceptability, they don't (and can't) guarantee anything like overall correctness.

Just checking off the list of things you've determined to be irrelevant. Compiler? Nope. Linter? Nope. Test suite? Nope. How about TLA+ specifications?

TLA+ specs don’t verify code. They verify design. Such design can be expressed in whatever, including pseudocode (think algorithms notation in textbooks). Then you write the TLA specs that will judge if invariants are truly respected. Once you’re sure of the design, you can go and implement it, but there’s no hard constraints like a type system.

Re: I read all of Cloudflare's Claude-generated commits

#137

Post by kentonv » Sat, Jun 07, 2025, 2:39 PM UTC

Fischgericht wrote: post #113

So, it means that you and the LLM together have managed to write SEVEN lines of trivial code per hour. On a protocol that is perfectly documented, where you can look at about one million other implementations when in doubt. It is not my intention to hurt your feelings, but it sounds like you and/or the LLM are not really good at their job. Looking at programmer salaries and LLM energy costs, this appears to be a very…

I spent about 5 days semi-focused on this codebase (though I always have lots of people interrupting me all the time). It's about 5000 lines (if you count comments, tests, and documentation, which you should). Where do you get 7 lines per hour?

Re: I read all of Cloudflare's Claude-generated commits

#138

Post by someothherguyy » Sat, Jun 07, 2025, 2:39 PM UTC

tptacek wrote: post #119

kiitos wrote: post #81

Earlier quoted context omitted.

> If the LLM hallucinates, then the code it produces is wrong. That wrong code isn't obviously or programmatically determinable as wrong, the agent has no way to figure out that it's wrong, it's not as if the LLM produces at the same time tests that identify that hallucinated code as being wrong. The only way that this wrong code can be identified as wrong is by the human user "looking closely" and figuring out that…

When you say "the LLM can easily hallucinate code that will satisfy the compiler but still fail the actual intent of the user", all you are saying is that the code will have bugs. My code has bugs. So does yours. You don't get to use the fancy word "hallucination" for reasonable-looking, readable code that compiles and lints but has bugs. I think at this point our respective points have been made, and we can wrap it…

Hallucination is a fancy word?

The parent seems to be, in part, referring to "reward hacking", which tends to be used as a super category to what many refer to as slop, hallucination, cheating, and so on.

https://courses.physics.illinois.edu/ece448/sp2025/slides/le...

Re: I read all of Cloudflare's Claude-generated commits

#139

Post by kentonv » Sat, Jun 07, 2025, 2:43 PM UTC

moron4hire wrote: post #117

I'm sorry, this all sounds like a fucking miserable experience. Like, if this is what my job becomes, I'll probably quit tech completely.

That's exactly what I thought, too, before I tried it!

Turns out it feels very different than I expected. I really recommend trying it rather than assuming. There's no learning curve, you just install Claude Code and run it in your repo and ask it for things.

(I am the author of the code being discussed. Or, uh, the author of the prompts at least.)

Re: I read all of Cloudflare's Claude-generated commits

#140

Post by kentonv » Sat, Jun 07, 2025, 2:49 PM UTC

IncreasePosts wrote: post #18

I asked this in the other thread (no response, but I was a bit late) How does anyone using AI like this have confidence that they aren't unintentionally plagiarizing code and violating the terms of whatever license it was released under? For random personal projects I don't see it mattering that much. But if a large corp is releasing code like this, one would hope they've done some due diligence that they have to jus…

I'm fairly confident that it's not just plagiarizing because I asked the LLM to implement a novel interface with unusual semantics. I then prompted for many specific fine-grain changes to implement features the way I wanted. It seems entirely implausible to me that there could exist prior art that happened to be structured exactly the way I requested.

Note that I came into this project believing that LLMs were plagiarism engines -- I was looking for that! I ended up concluding that this view was not consistent with the output I was actually seeing.