Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I read all of Cloudflare's Claude-generated commits

maxemitchell.com

Post reply on HN
Print view View original

251–260 of 291 posts

Re: I read all of Cloudflare's Claude-generated commits

#251

Post by steveklabnik »

ramchip wrote: post #211
kiitos wrote: post #158

Earlier quoted context omitted.

> one degree removed You're letting Claude do your programming for you, and then sweeping up whatever it does afterwards. Bluntly, you're off-loading your cognition to the machine. If that's fine by you then that's fine enough, it just means that the quality of your work becomes a function of your tooling rather than your capabilities.

Personally, I spend _more_ time thinking with Claude. I can focus on the design decisions while it does the mechanical work of turning that into code. Sometimes I give the agent a vague design ("make XYZ configurable") and it implements it the wrong way, so I'll tell it to do it again with more precise instructions ("use a config file instead of a CLI argument"). The best thing is you can tell it after it wrote 500 l…

I was recently considering a refactoring in a work codebase. I had an interesting discussion with Claude about the tradeoffs, then had it show me what the code would look like after the refactor, both in a very simple case, and in one of the most complex cases. All of this informed what path I ended up taking, but especially the real-world examples meant this was a much better informed decision than just "hmm, yeah seems like it would be a lot of work but also probably worth it."

Re: I read all of Cloudflare's Claude-generated commits

#254

Post by wat10000 »

lelanthran wrote: post #190
cesarb wrote: post #182

Earlier quoted context omitted.

> Compilers are deterministic. Given the same input you always get the same output Except when they aren't. See for instance https://gcc.gnu.org/onlinedocs/gcc-15.1.0/gcc/Developer-Opti... or the __DATE__/__TIME__ macros.

From the link: > You can use the -frandom-seed option to produce reproducibly identical object files. Deterministic. Also, with regard to __DATE__/__TIME__ macros, those are deterministic, because the current date and time are part of the inputs.

Determinism is predicated on what you consider to be the relevant inputs.

Many compilers are not deterministic when only considering the source files or even the current time. For example, any output produced by iterating over a hash table with pointer keys is likely to depend on ASLR and thus be nondetermistic unless you consider the ASLR randomization to be one of the inputs. Any output that depends on directory iteration order is likely to be consistent on a single computer but vary across computers.

LLMs aren’t magic. They’re software running on inputs like anything else, which means they’re deterministic if you constrain all the inputs.

Re: I read all of Cloudflare's Claude-generated commits

#255

Post by hooverd »

starkparker wrote: post #203

> Almost every feature required multiple iterations and refinements. This isn't a limitation—it's how the collaboration works. I guess that's where a big miss in understanding so much of the messaging about generative AI in coding happens for me, and why the Fly.io skepticism blog post irritated me so much as well. It _is_ how collaboration with a person works, but the when you have to fix the issues that the tool cr…

Your table saw hungers for fingers.

Re: I read all of Cloudflare's Claude-generated commits

#256

Post by nothrabannosir »

layer8 wrote: post #230
maxemitchell wrote: post #146

Earlier quoted context omitted.

I did human notes -> had Claude condense and edit -> manually edit. A few of the sentences (like the stinky one below) were from Claude which I kept if it matched my own thoughts, though most were changed for style/prose. I'm still experimenting with it. I find it can't match style at all, and even with the manual editing it still "smells like AI" as you picked up. But, it also saves time. My prompt was essentially "…

I wonder if those notes wouldn’t have been more interesting as-is, and possibly also more condensed.

I wish there were a way to opt-out of LLM generated text and see the prompt. In any context. It's always more informative, more human, more memorable, more accurate, and more representative of what the author was actually trying to convey.

Re: I read all of Cloudflare's Claude-generated commits

#257

Post by kiitos »

kentonv wrote: post #207
kiitos wrote: post #196

Earlier quoted context omitted.

> I'm still deciding what gets built and how it is designed, which is the interesting part. How, exactly? Do you think that you're "deciding what gets built and how it's designed" by iterating on the prompts that you feed to the LLM that generates the code? Or are you saying that you're somehow able to write the "interesting" code, and can instruct the LLM to generate the "boring and obvious" code that needs to be fi…

Did you actually read the commit history? My prompts specify very precisely what should be implemented. I specified the public API and high-level design upfront. I let the AI come up with its own storage schema initially but then I prompted it very specifically through several improvements (e.g. "denormalize this table into this other table to eliminate a lookup"). I designed the end-to-end encryption scheme and told…

Right, so -- 'you think that you're "deciding what gets built and how it's designed" by iterating on the prompts that you feed to the LLM that generates the code'

> My prompts specify very precisely what should be implemented.

And the precision of your prompt's specifications, has no reliable impact on exactly what code the LLM returns as output.

> With the details I provided, combined with the OAuth spec, there was really very little room left for any creativity in the code. It was basically connect-the-dots at that point.

I truly don't know how you can come to this conclusion, if you have any amount of observed experience with any of the current-gen LLM tools. No amount of prompt engineering gets you a reliable mapping from input query to output code.

> I designed the end-to-end encryption scheme and told it in detail how to implement it. I pointed out bugs and explained how to fix them. And so on.

I guess my response here is that, if you think that this approach to prompt engineering gets you a generated code result that is in any sense equivalent, or even comparable, in terms of quality, to the work that you could produce yourself, as a professional and senior-level software engineer, then, man, we're on different planets. Pointing out bugs and explaining how to fix them in your prompts in no way gets you deterministic, reliable, accurate, high-quality code as output. And actually forget about high-quality, I mean even just bare minimum table-stakes requirements-satisfying stuff.. !

Re: I read all of Cloudflare's Claude-generated commits

#258

Post by never_inline »

isaacremuant wrote: post #245
latexr wrote: post #87

Earlier quoted context omitted.

Can we please stop using the em-dash as a metric to “detect” LLM writing? It’s lazy and wrong. Plenty of people use em-dashes, it’s a useful punctuation mark . If humans didn’t use them, they wouldn’t be in the LLM training data. There are better clues, like the kind of vague pretentious babble bad marketers use to make their products and ideas seem more profound than they are. It’s a type of bad writing which looks…

It's not lazy and wrong. It's a fantastic indicator. > If humans didn’t use them, they wouldn’t be in the LLM training data. Humans weren't using them in every context as they are now. They might've been used in books but blog posts and work documents weren't full of them. It's not a definite thing but it's absolutely a good indicator.

They may be overrepresented in the RLHF

Re: I read all of Cloudflare's Claude-generated commits

#259

Post by tptacek »

kiitos wrote: post #257
kentonv wrote: post #207

Earlier quoted context omitted.

Did you actually read the commit history? My prompts specify very precisely what should be implemented. I specified the public API and high-level design upfront. I let the AI come up with its own storage schema initially but then I prompted it very specifically through several improvements (e.g. "denormalize this table into this other table to eliminate a lookup"). I designed the end-to-end encryption scheme and told…

Right, so -- 'you think that you're "deciding what gets built and how it's designed" by iterating on the prompts that you feed to the LLM that generates the code' > My prompts specify very precisely what should be implemented. And the precision of your prompt's specifications, has no reliable impact on exactly what code the LLM returns as output. > With the details I provided, combined with the OAuth spec, there was…

Nobody has claimed to be getting deterministic outputs from LLMs.

Re: I read all of Cloudflare's Claude-generated commits

#260

Post by kiitos »

tptacek wrote: post #259
kiitos wrote: post #257

Earlier quoted context omitted.

Right, so -- 'you think that you're "deciding what gets built and how it's designed" by iterating on the prompts that you feed to the LLM that generates the code' > My prompts specify very precisely what should be implemented. And the precision of your prompt's specifications, has no reliable impact on exactly what code the LLM returns as output. > With the details I provided, combined with the OAuth spec, there was…

Nobody has claimed to be getting deterministic outputs from LLMs.

> My prompts specify very precisely what should be implemented. I specified the public API and high-level design upfront. I let the AI come up with its own storage schema initially but then I prompted it very specifically through several improvements (e.g. "denormalize this table into this other table to eliminate a lookup"). I designed the end-to-end encryption scheme and told it in detail how to implement it. I pointed out bugs and explained how to fix them. And so on.

OK. Replace "[expected] deterministic output" with whatever term best fits what this block of text is describing, as that's what I'm talking about. The claim is that a sufficiently-precisely-specified prompt can produce reliably-correct code. Which is just clearly not the case, as of today.

Post reply on HN