Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I read all of Cloudflare's Claude-generated commits

maxemitchell.com

Post reply on HN
Print view View original

191–200 of 291 posts

Re: I read all of Cloudflare's Claude-generated commits

#191

Post by kasey_junk »

lcnPylGDnU4H9OF wrote: post #145
kiitos wrote: post #57

Earlier quoted context omitted.

What on earth are you talking about?? If the LLM hallucinates, then the code it produces is wrong. That wrong code isn't obviously or programmatically determinable as wrong, the agent has no way to figure out that it's wrong, it's not as if the LLM produces at the same time tests that identify that hallucinated code as being wrong. The only way that this wrong code can be identified as wrong is by the human user "loo…

> You seem to have this fundamental belief that the code that's produced by your LLM is valid and doesn't need to be evaluated, line-by-line, by a human, before it can be committed?? This is a mistaken understanding. The person you responded to has written on these thoughts already and they used memorable words in response to this proposal: > Are you a vibe coding Youtuber? Can you not read code? If so: astute point.…

We should go one step past this and come up with an industry practice where we get someone other than the author to read the code before we merge it.

Re: I read all of Cloudflare's Claude-generated commits

#192

Post by tptacek »

kiitos wrote: post #180
tptacek wrote: post #176

Earlier quoted context omitted.

I'm sure there is when you're just getting your sea legs in some environment, but at some point most of the code you write in a given environment is rote. Rote code is both depleting and mutagenic --- if you're fluent and also interested in programming, you'll start convincing yourself to do stupid stuff to make the code less rote ("DRY it up", "make a DSL", &c) that makes your code less readable and maintainable. It…

> but at some point most of the code you write in a given environment is rote "Most of the code one writes in a given environment is rote" is true in the same sense that most of the words one writes in any given bit of text are rote e.g. conjunctions, articles, prepositions, etc.

Some writers I know are convinced this is true, but I still don't think the comparison is completely apt, because deliberately rote code with modulated expressiveness is often (even usually) a virtue in coding, and not always so with writing. For experienced or enthusiastic coders, that is to say, the effort is often in not doing stupid stuff to make the code more clever.

Straight-line replacement-grade mid code that just does the things a prompt tells it to in the least clever most straightforward way possible is usually a good thing; that long clunky string of modifiers goes by the name "maintainability".

Re: I read all of Cloudflare's Claude-generated commits

#193

Post by tptacek »

kentonv wrote: post #179
kiitos wrote: post #158

Earlier quoted context omitted.

> one degree removed You're letting Claude do your programming for you, and then sweeping up whatever it does afterwards. Bluntly, you're off-loading your cognition to the machine. If that's fine by you then that's fine enough, it just means that the quality of your work becomes a function of your tooling rather than your capabilities.

I don't agree. The AI largely does the boring and obvious parts. I'm still deciding what gets built and how it is designed, which is the interesting part.

It's the same with me, with the added wrinkle of pulling each PR branch down and refactoring things (and, ironically, introducing my own bugs).

Re: I read all of Cloudflare's Claude-generated commits

#194

Post by singhrac »

stingraycharles wrote: post #37
fastball wrote: post #30

Earlier quoted context omitted.

The idea as stated is a poor one, but a slight reshuffling and it seems promising: You generate code with LLMs. You write tests for this code, either using LLMs or on your own. You of course commit your actual code: it is required to actually run the program, after all. However you also save the entire prompt chain somewhere. Then (as stated in the article), when a much better model comes along, you re-run that chain…

Means the temperature should be set to 0 (which not every provider supports) so that the output becomes entirely deterministic. Right now with most models if you give the same input prompt twice it will give two different solutions.

Production inference is not deterministic because of sharding (i.e. parameter weights on several GPUs on the same machine or MoE), timing-based kernel choices (e.g. torch.backends.cudnn.benchmark), or batched routing in MoEs. Probably best to host a small model yourself.

Re: I read all of Cloudflare's Claude-generated commits

#195

Post by axi0m »

>> what if we treated prompts as the actual source code?

And they probably will be. Looks like prompts have become the new higher-level coding language, the same way JavaScript is a human-friendly abstraction of an existing programming language (like C), which is already a more accessible way to write assembly itself, and the same goes for the underlying binary code... I guess we eventually reached the final step in the development chain, bridging the gap between hardware instructions and human language.

Re: I read all of Cloudflare's Claude-generated commits

#196

Post by kiitos »

kentonv wrote: post #179
kiitos wrote: post #158

Earlier quoted context omitted.

> one degree removed You're letting Claude do your programming for you, and then sweeping up whatever it does afterwards. Bluntly, you're off-loading your cognition to the machine. If that's fine by you then that's fine enough, it just means that the quality of your work becomes a function of your tooling rather than your capabilities.

I don't agree. The AI largely does the boring and obvious parts. I'm still deciding what gets built and how it is designed, which is the interesting part.

> I'm still deciding what gets built and how it is designed, which is the interesting part.

How, exactly? Do you think that you're "deciding what gets built and how it's designed" by iterating on the prompts that you feed to the LLM that generates the code?

Or are you saying that you're somehow able to write the "interesting" code, and can instruct the LLM to generate the "boring and obvious" code that needs to be filled-in to make your interesting code work? (This is certainly not what's indicated by your commit history, but, who knows?)

Re: I read all of Cloudflare's Claude-generated commits

#198

Post by tptacek »

saagarjha wrote: post #96
gizmo686 wrote: post #59

Earlier quoted context omitted.

My work has involved a project that is almost entirely generated code for over a decade. Not AI generated, the actual work of the project is in creating the code generator. One of the things we learned very quickly was that having generated source code in the same repository as actual source code was not sustainable. The nature of reviewing changes is just too different between them. Another thing we learned very qui…

I think the biggest difference here is that your code generator is probably deterministic and you likely are able to debug the results it produces rather than treating it like a black box.

People keep saying this and it doesn't make sense. I review code. I don't construct a theory of mind of the author of the code. With AI-generated code, if it isn't eminently reviewable, I reflexively kill the PR and either try again or change the tasking.

There's always this vibe that, like, AI code is like an IOCCC puzzle. No. It's extremely boring mid-code. Any competent developer can review it.

Re: I read all of Cloudflare's Claude-generated commits

#199

Post by tptacek »

lelanthran wrote: post #110
saagarjha wrote: post #95

Earlier quoted context omitted.

I feel like using a compiler is in a sense a code generator where you don't commit the actual output

> I feel like using a compiler is in a sense a code generator where you don't commit the actual output Compilers are deterministic. Given the same input you always get the same output so there's no reason to store the output. If you don't get the same output we call it a compiler bug! LLMs do not work this way. (Aside: Am I the only one who feels that the entire AI industry is predicated on replacing only development…

Why does it matter to you if the code generator is deterministic? The code is.

If LLM generation was like a Makefile step, part of your build process, this concern would make a lot of sense. But nobody, anywhere, does that.

Re: I read all of Cloudflare's Claude-generated commits

#200

Post by buu700 »

tptacek wrote: post #198
saagarjha wrote: post #96

Earlier quoted context omitted.

I think the biggest difference here is that your code generator is probably deterministic and you likely are able to debug the results it produces rather than treating it like a black box.

People keep saying this and it doesn't make sense. I review code. I don't construct a theory of mind of the author of the code. With AI-generated code, if it isn't eminently reviewable, I reflexively kill the PR and either try again or change the tasking. There's always this vibe that, like, AI code is like an IOCCC puzzle. No. It's extremely boring mid-code. Any competent developer can review it.

I assumed they were describing AI itself as a black box (contrasting it with deterministic code generation), not the output of AI.
Post reply on HN