Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I read all of Cloudflare's Claude-generated commits

maxemitchell.com

Post reply on HN
Print view View original

241–250 of 291 posts

Re: I read all of Cloudflare's Claude-generated commits

#241

Post by kasey_junk »

lcnPylGDnU4H9OF wrote: post #208
kasey_junk wrote: post #191

Earlier quoted context omitted.

We should go one step past this and come up with an industry practice where we get someone other than the author to read the code before we merge it.

I don’t understand your point. Are you saying that it sounds like that wouldn’t happen?

I’m being sarcastic. The person you are responding to is implying that reading code carefully before merging it is some daunting or new challenge. In fact it’s been standard practice in our industry for 2 or more people to do that as a matter of course.

Re: I read all of Cloudflare's Claude-generated commits

#242

Post by JackSlateur »

_pdp_ wrote: post #188

I commented on the original discussion a few days ago but I will do it again. Why is this such a big deal? This library is not even that interesting. It is very straightforward task I expect most programers will be able to pull off easily. 2/3 of the code is type interfaces and comments. The rest is by book implementation of a protocol that is not even that complex. Please, there are some React JSX files in your code…

Of course, this is a pathetic commercial, nothing serious

As you say, the code is not interesting, it deals with a well known topic

And it required lots of man power to get done

tldr: this is a non-event disguised as incredible success. No doubt cloudflare is making money with that AI crap, somehow.

Re: I read all of Cloudflare's Claude-generated commits

#243

Post by paxys »

SrslyJosh wrote: post #5

> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…

Forget different model versions. The exact same model with the exact same prompt will generate vastly different code each subsequent time you invoke it.

Re: I read all of Cloudflare's Claude-generated commits

#244

Post by isaacremuant »

starkparker wrote: post #203

> Almost every feature required multiple iterations and refinements. This isn't a limitation—it's how the collaboration works. I guess that's where a big miss in understanding so much of the messaging about generative AI in coding happens for me, and why the Fly.io skepticism blog post irritated me so much as well. It _is_ how collaboration with a person works, but the when you have to fix the issues that the tool cr…

Likewise when they use all these benchmarks for "intelligence" and the tool will do the silliest things that you'd consider unacceptable from a person once you've told them a few times not to do a certain thing.

I love the paradigm shift but hate when the hype is uninformed or dishonest or not treating it with an eye for quality.

Re: I read all of Cloudflare's Claude-generated commits

#245

Post by isaacremuant »

latexr wrote: post #87
ZephyrBlu wrote: post #85

Earlier quoted context omitted.

Also: " This OAuth library represents something larger than a technical milestone—it's evidence of a new creative dynamic emerging " Em-dash baby.

Can we please stop using the em-dash as a metric to “detect” LLM writing? It’s lazy and wrong. Plenty of people use em-dashes, it’s a useful punctuation mark . If humans didn’t use them, they wouldn’t be in the LLM training data. There are better clues, like the kind of vague pretentious babble bad marketers use to make their products and ideas seem more profound than they are. It’s a type of bad writing which looks…

It's not lazy and wrong. It's a fantastic indicator.

> If humans didn’t use them, they wouldn’t be in the LLM training data.

Humans weren't using them in every context as they are now. They might've been used in books but blog posts and work documents weren't full of them.

It's not a definite thing but it's absolutely a good indicator.

Re: I read all of Cloudflare's Claude-generated commits

#246

Post by daveguy »

tptacek wrote: post #198
saagarjha wrote: post #96

Earlier quoted context omitted.

I think the biggest difference here is that your code generator is probably deterministic and you likely are able to debug the results it produces rather than treating it like a black box.

People keep saying this and it doesn't make sense. I review code. I don't construct a theory of mind of the author of the code. With AI-generated code, if it isn't eminently reviewable, I reflexively kill the PR and either try again or change the tasking. There's always this vibe that, like, AI code is like an IOCCC puzzle. No. It's extremely boring mid-code. Any competent developer can review it.

You construct a theory of mind of the author of a work whether you recognize you are doing it or not. There are certain things everyone assumes about code based on the fact that we expect someone who writes code to have simple common sense. Which, of course, LLMs do not.

When you are talking to a person and interpreting what they mean, you have an inherent theory of mind whether you are consciously thinking "how does this person think" or not. It's how we communicate with other people efficiently and it's one of the many things missing with LLM roulette. It's not that you generate a new "theory of mind" with every interaction. It's not something you have to consciously do (although you can, like breathing).

Re: I read all of Cloudflare's Claude-generated commits

#247

Post by dragonwriter »

SrslyJosh wrote: post #5

> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…

Also, while it is in principle possible to have a deterministic LLM, the ones used by coding assistants aren't deterministic, so the prompts would not reliably reproduce the same software.

There is definitely an argument, for also committing prompts, but it makes no sense to only commit prompts.

Re: I read all of Cloudflare's Claude-generated commits

#248

Post by dragonwriter »

layer8 wrote: post #223
diggan wrote: post #82

Earlier quoted context omitted.

If you can 100% reproduce the same generated code from the same prompts, even 5 years later, given the same versions and everything then I'd say "Sure, go ahead and don't saved the generated code, we can always regenerate it". As someone who spent some time in frontend development, we've been doing it like that for a long time with (MB+) generated code, keeping it in scm just isn't feasible long-term. But given this…

Temperature > 0 isn’t a problem as long as you can specify/save the random seed and everything else is deterministic. Of course, “as long as” is still a tall order here.

My understanding is that the implementation of modern hosted LLMs is nondeterministic even with known seed because the generated results are sensitive to a number of other factors including, but not limited to, other prompts running in the same batch.

Re: I read all of Cloudflare's Claude-generated commits

#249

Post by tptacek »

rawling wrote: post #239
tptacek wrote: post #199

Earlier quoted context omitted.

Why does it matter to you if the code generator is deterministic? The code is. If LLM generation was like a Makefile step, part of your build process, this concern would make a lot of sense. But nobody, anywhere, does that.

> If LLM generation was like a Makefile step, part of your build process, this concern would make a lot of sense. But nobody, anywhere, does that. Top level comment of this thread, quoting the article: > Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting im…

Ohhhhhhh. Thanks for clearing this up for me. I felt like I was going a little crazy (because, having missed that part of the thread, I sort of was). Appreciated!

Re: I read all of Cloudflare's Claude-generated commits

#250

Post by tptacek »

layer8 wrote: post #222
tptacek wrote: post #204

Earlier quoted context omitted.

Right, you definitely can't do that. People do talk as if the question was whether we could stick LLM calls into Makefiles. Nobody would ever do that, at least not with the technology we have at hand.

But that’s exactly what the part of the article quoted in the root comment ( https://news.ycombinator.com/item?id=44206758 ) is about, and in reference to future AI technology. That’s was this subthread was discussing.

Yep, you're 100% right. Sorry!
Post reply on HN