Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I read all of Cloudflare's Claude-generated commits

maxemitchell.com

Post reply on HN
Print view View original

21–30 of 291 posts

Re: I read all of Cloudflare's Claude-generated commits

#21

Post by double0jimb0 »

groby_b wrote: post #10
SupremumLimit wrote: post #4

It's an interesting review but I really dislike this type of techno-utopian determinism: "When models inevitably improve..." Says who? How is it inevitable? What if they've actually reached their limits by now?

It is "inevitable" in the sense that in 99% of the cases, tomorrow is just like yesterday. LLMs have been continually improving for years now. The surprising thing would be them not improving further. And if you follow the research even remotely, you know they'll improve for a while, because not all of the breakthroughs have landed in commercial models yet. It's not "techno-utopian determinism". It's a clearly visibl…

Are there places that follow the research that speak to the layperson?

Re: I read all of Cloudflare's Claude-generated commits

#22

Post by declan_roberts »

These posts are funny to me because prompt engineers point at them as evidence of the fast-approaching software engineer obsolescence but the years of experience in software engineering necessary to even guide an AI in this way is very high.

The reason he keeps adjusting the prompts is because he knows how to program. He knows what it should look like.

It just blurs the line between engineer and tool.

Re: I read all of Cloudflare's Claude-generated commits

#24

Post by saghm »

IncreasePosts wrote: post #18

I asked this in the other thread (no response, but I was a bit late) How does anyone using AI like this have confidence that they aren't unintentionally plagiarizing code and violating the terms of whatever license it was released under? For random personal projects I don't see it mattering that much. But if a large corp is releasing code like this, one would hope they've done some due diligence that they have to jus…

Safety in the shadow of giant tech companies. People were upset when Microsoft released Copilot trained on GitHub data, but nobody who cared doing do anything about it, and nobody who could have done something about it cared, so it just became the new norm.

Re: I read all of Cloudflare's Claude-generated commits

#25

Post by greyadept »

Sevii wrote: post #8
SupremumLimit wrote: post #4

It's an interesting review but I really dislike this type of techno-utopian determinism: "When models inevitably improve..." Says who? How is it inevitable? What if they've actually reached their limits by now?

Models have improved significantly over the last 3 months. Yet people have been saying 'What if they've actually reached their limits by now?' for pushing 3 years.

For me, improvement means no hallucination, but that only seems to have gotten worse and I'm interested to find out whether it's actually solvable at all.

Re: I read all of Cloudflare's Claude-generated commits

#26

Post by [deleted] »

rectang wrote: post #11
SrslyJosh wrote: post #5

> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…

>> what if we treated prompts as the actual source code? You would not do this because: unlike programming languages, natural languages are ambiguous and thus inadequate to fully specify software.

[deleted]

Re: I read all of Cloudflare's Claude-generated commits

#27

Post by tayo42 »

SrslyJosh wrote: post #5

> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…

I'm pretty sure most people aren't doing "software engineering" when they program. There's the whole world of WordPress and dream Weaver like programing out there too where the consequences of messing up aren't really important.

Llms can be configured to have deterministic output too

Re: I read all of Cloudflare's Claude-generated commits

#28

Post by dymk »

greyadept wrote: post #25
Sevii wrote: post #8

Earlier quoted context omitted.

Models have improved significantly over the last 3 months. Yet people have been saying 'What if they've actually reached their limits by now?' for pushing 3 years.

For me, improvement means no hallucination, but that only seems to have gotten worse and I'm interested to find out whether it's actually solvable at all.

All the benchmarks would disagree with you

Re: I read all of Cloudflare's Claude-generated commits

#29

Post by thuuuomas »

dymk wrote: post #28
greyadept wrote: post #25

Earlier quoted context omitted.

For me, improvement means no hallucination, but that only seems to have gotten worse and I'm interested to find out whether it's actually solvable at all.

All the benchmarks would disagree with you

Today’s public benchmarks are yesterday’s training data.

Re: I read all of Cloudflare's Claude-generated commits

#30

Post by fastball »

SrslyJosh wrote: post #5

> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…

The idea as stated is a poor one, but a slight reshuffling and it seems promising:

You generate code with LLMs. You write tests for this code, either using LLMs or on your own. You of course commit your actual code: it is required to actually run the program, after all. However you also save the entire prompt chain somewhere. Then (as stated in the article), when a much better model comes along, you re-run that chain, presumably with prompting like "create this project, focusing on efficiency" or "create this project in Rust" or "create this project, focusing on readability of the code". Then you run the tests against the new codebase and if the suite passes you carry on, with a much improved codebase. The theoretical benefit of this over just giving your previously generated code to the LLM and saying "improve the readability" is that the newer (better) LLM is not burdened by the context of the "worse" decisions made by the previous LLM.

Obviously it's not actually that simple, as tests don't catch everything (tho with fuzz testing and complete coverage and such they can catch most issues), but we programmers often treat them as if they do, so it might still be a worthwhile endeavor.

Post reply on HN