Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I read all of Cloudflare's Claude-generated commits

maxemitchell.com

Post reply on HN
Print view View original

201–210 of 291 posts

Re: I read all of Cloudflare's Claude-generated commits

#201

Post by tptacek »

buu700 wrote: post #200
tptacek wrote: post #198

Earlier quoted context omitted.

People keep saying this and it doesn't make sense. I review code. I don't construct a theory of mind of the author of the code. With AI-generated code, if it isn't eminently reviewable, I reflexively kill the PR and either try again or change the tasking. There's always this vibe that, like, AI code is like an IOCCC puzzle. No. It's extremely boring mid-code. Any competent developer can review it.

I assumed they were describing AI itself as a black box (contrasting it with deterministic code generation), not the output of AI.

Right, I get that, and an LLM call by itself clearly is a black box. I just don't get why that's supposed to matter. It produces an artifact I can (and must) verify myself.

Re: I read all of Cloudflare's Claude-generated commits

#202

Post by buu700 »

tptacek wrote: post #201
buu700 wrote: post #200

Earlier quoted context omitted.

I assumed they were describing AI itself as a black box (contrasting it with deterministic code generation), not the output of AI.

Right, I get that, and an LLM call by itself clearly is a black box. I just don't get why that's supposed to matter. It produces an artifact I can (and must) verify myself.

Because if the LLM is a black box and its output must ultimately be verified by humans, then you can't treat conversion of prompts into code as a simple build step as though an AI agent were just some sort of compiler. You still need to persist the actual code in source control.

(I assume that isn't what you're actually arguing against, in which case at least one of us must have misread something from the parent chain.)

Re: I read all of Cloudflare's Claude-generated commits

#203

Post by starkparker »

> Almost every feature required multiple iterations and refinements. This isn't a limitation—it's how the collaboration works.

I guess that's where a big miss in understanding so much of the messaging about generative AI in coding happens for me, and why the Fly.io skepticism blog post irritated me so much as well.

It _is_ how collaboration with a person works, but the when you have to fix the issues that the tool created, you aren't collaborating with a person, you're making up for a broken tool.

I can't think of any field where I'd be expected to not only put up with, but also celebrate, a tool that screwed up and required manual intervention so often.

The level of anthropomorphism that occurs in order to advocate on behalf of generative AI use leads to saying things like "it's how collaboration works" here, when I'd never say the same thing about the table saw in my woodshop, or even the relatively smart cruise control on my car.

Generative AI is still just a tool built by people following a design, and which purportedly makes work easier. But when my saw tears out cuts that I have to then sand or recut, or when my car slams on the brakes because it can't understand a bend in the road around a parking lane, I don't shrug and ascribe them human traits and blame myself for being frustrated over how they collaborate with me.

Re: I read all of Cloudflare's Claude-generated commits

#204

Post by tptacek »

buu700 wrote: post #202
tptacek wrote: post #201

Earlier quoted context omitted.

Right, I get that, and an LLM call by itself clearly is a black box. I just don't get why that's supposed to matter. It produces an artifact I can (and must) verify myself.

Because if the LLM is a black box and its output must ultimately be verified by humans, then you can't treat conversion of prompts into code as a simple build step as though an AI agent were just some sort of compiler. You still need to persist the actual code in source control. (I assume that isn't what you're actually arguing against, in which case at least one of us must have misread something from the parent chai…

Right, you definitely can't do that. People do talk as if the question was whether we could stick LLM calls into Makefiles. Nobody would ever do that, at least not with the technology we have at hand.

Re: I read all of Cloudflare's Claude-generated commits

#206

Post by ramchip »

latexr wrote: post #111
grey-area wrote: post #107

Earlier quoted context omitted.

Very few people use n dashes in internet writing as opposed to dashes as they are not available on the default keyboard.

That’s not true at all. Apple’s OS by default have smart punctuation enabled and convert -- (two hyphens) into — (“em-dash”; not an “en-dash”, which has a different purpose), " " (dumb quotes) into “ ” (smart quotes), and so forth. Furthermore, on macOS there are simple key combinations (e.g. with ⌥) to make all sort of smart punctuation even if you don’t have the feature enabled by default, and on iOS you can long p…

> assuming a single character immediately means they used an LLM is just plain wrong

I don't see anyone doing that here. LLM writing was brought up because of the writing style, not the dash. It just reinforces the suspicion.

Re: I read all of Cloudflare's Claude-generated commits

#207

Post by kentonv »

kiitos wrote: post #196
kentonv wrote: post #179

Earlier quoted context omitted.

I don't agree. The AI largely does the boring and obvious parts. I'm still deciding what gets built and how it is designed, which is the interesting part.

> I'm still deciding what gets built and how it is designed, which is the interesting part. How, exactly? Do you think that you're "deciding what gets built and how it's designed" by iterating on the prompts that you feed to the LLM that generates the code? Or are you saying that you're somehow able to write the "interesting" code, and can instruct the LLM to generate the "boring and obvious" code that needs to be fi…

Did you actually read the commit history?

My prompts specify very precisely what should be implemented. I specified the public API and high-level design upfront. I let the AI come up with its own storage schema initially but then I prompted it very specifically through several improvements (e.g. "denormalize this table into this other table to eliminate a lookup"). I designed the end-to-end encryption scheme and told it in detail how to implement it. I pointed out bugs and explained how to fix them. And so on.

All the thinking happened in those prompts. With the details I provided, combined with the OAuth spec, there was really very little room left for any creativity in the code. It was basically connect-the-dots at that point.

Re: I read all of Cloudflare's Claude-generated commits

#208

Post by lcnPylGDnU4H9OF »

kasey_junk wrote: post #191
lcnPylGDnU4H9OF wrote: post #145

Earlier quoted context omitted.

> You seem to have this fundamental belief that the code that's produced by your LLM is valid and doesn't need to be evaluated, line-by-line, by a human, before it can be committed?? This is a mistaken understanding. The person you responded to has written on these thoughts already and they used memorable words in response to this proposal: > Are you a vibe coding Youtuber? Can you not read code? If so: astute point.…

We should go one step past this and come up with an industry practice where we get someone other than the author to read the code before we merge it.

I don’t understand your point. Are you saying that it sounds like that wouldn’t happen?

Re: I read all of Cloudflare's Claude-generated commits

#209

Post by mastazi »

> Around the 40-commit mark, manual commits became frequent—styling, removing unused methods, the kind of housekeeping that coding models still struggle with. It's clear that AI generated >95% of the code, but human oversight was essential throughout.

But things like styling and unused code removal have been automated for a long time already, thanks to non-AI tools; assuming that the AI agent has access to those tools (e.g. assuming the agent can trigger a linter), then the engineer could have just included these steps in the prompts instead of running them manually.

EDIT - I still think there are aspects where AI is obviously lacking, I just think those specific examples are not among them

Re: I read all of Cloudflare's Claude-generated commits

#210

Post by buu700 »

mastazi wrote: post #209

> Around the 40-commit mark, manual commits became frequent—styling, removing unused methods, the kind of housekeeping that coding models still struggle with. It's clear that AI generated >95% of the code, but human oversight was essential throughout. But things like styling and unused code removal have been automated for a long time already, thanks to non-AI tools; assuming that the AI agent has access to those tool…

Speaking of which, something funny I've noticed when using agents with prettier in a pre-commit hook is that the logs occasionally include the model thanking "me" for cleaning up its code formatting.
Post reply on HN