151–160 of 291 posts
Earlier quoted context omitted.
The sequence of commits talked about by the OP -- i.e. kenton's coding session's commits -- are like one degree removed from 100% pure vibe coding.
Your claim here being that Kenton Varda isn't reading the code he's generating. Got it. Good note.
Earlier quoted context omitted.
> If the LLM hallucinates, then the code it produces is wrong. That wrong code isn't obviously or programmatically determinable as wrong, the agent has no way to figure out that it's wrong, it's not as if the LLM produces at the same time tests that identify that hallucinated code as being wrong. The only way that this wrong code can be identified as wrong is by the human user "looking closely" and figuring out that…
When you say "the LLM can easily hallucinate code that will satisfy the compiler but still fail the actual intent of the user", all you are saying is that the code will have bugs. My code has bugs. So does yours. You don't get to use the fancy word "hallucination" for reasonable-looking, readable code that compiles and lints but has bugs. I think at this point our respective points have been made, and we can wrap it…
There is an obvious and categorical difference between the "bugs" that an LLM produces as part of its generated code, and the "bugs" that I produce as part of the code that I write. You don't get to conflate these two classes of bugs as though they are equivalent, or even comparable. They aren't.
Earlier quoted context omitted.
Irrelevant, really. Tests establish a minimum threshold of acceptability, they don't (and can't) guarantee anything like overall correctness.
Just checking off the list of things you've determined to be irrelevant. Compiler? Nope. Linter? Nope. Test suite? Nope. How about TLA+ specifications?
Earlier quoted context omitted.
Your claim here being that Kenton Varda isn't reading the code he's generating. Got it. Good note.
No, that's not at all my claim, as it's obvious from the commit history that Kenton is reading the code he's generating before committing it.
> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation. Please god, no, never do this. For one thing, why would you not commit the generated source code when storage is essentially free? That seems insane for multiple reasons. > When models ine…
My work has involved a project that is almost entirely generated code for over a decade. Not AI generated, the actual work of the project is in creating the code generator. One of the things we learned very quickly was that having generated source code in the same repository as actual source code was not sustainable. The nature of reviewing changes is just too different between them. Another thing we learned very qui…
My rule of the thumb is to have both in same repo, but treat generated code like binary data. This was informed by when I was burned by a tooling regression that broke the generated code and the investigation was complicated by having to correlate commits across different repositories
Many of you are failing to conprehend the potential scale of AI generated codebases. Take note - there is no limit. Every feature you or the AI can prompt can be generated. Imagine if you were immortal and given unlimited storage. Imagine what you could create. That’s a prompt away. Even now you’re still restricting your thinking to the old ways.
You’re sounding like a religious zealot recruiting for a cult. No, it is not possible to prompt every feature, and I suspect people who believe LLMs can accurately program anything in any language are frankly not solving any truly novel or interesting problems, because if they were they’d see the obvious cracks.
The vast majority of problems in programming aren't novel or interesting.
So, it means that you and the LLM together have managed to write SEVEN lines of trivial code per hour. On a protocol that is perfectly documented, where you can look at about one million other implementations when in doubt. It is not my intention to hurt your feelings, but it sounds like you and/or the LLM are not really good at their job. Looking at programmer salaries and LLM energy costs, this appears to be a very…
Here's their response
>It took me a few days to build the library with AI.
>I estimate it would have taken a few weeks, maybe months to write by hand.
>That said, this is a pretty ideal use case: implementing a well-known standard on a well-known platform with a clear API spec.
https://news.ycombinator.com/item?id=44160208
Lines of code per hour is a terrible metric to use. Additionally, it's far easier to critique code that's already written!
Earlier quoted context omitted.
No, that's not at all my claim, as it's obvious from the commit history that Kenton is reading the code he's generating before committing it.
What do you mean by "one degree removed from 100% pure vibe coding", then? The definition of vibe coding is letting the AI code without review...
You're letting Claude do your programming for you, and then sweeping up whatever it does afterwards. Bluntly, you're off-loading your cognition to the machine. If that's fine by you then that's fine enough, it just means that the quality of your work becomes a function of your tooling rather than your capabilities.
Earlier quoted context omitted.
> One of the things we learned very quickly was that having generated source code in the same repository as actual source code was not sustainable. Keeping a repository with the prompts, or other commands separate is fine, but not committing the generated code at all I find questionable at best.
If you can 100% reproduce the same generated code from the same prompts, even 5 years later, given the same versions and everything then I'd say "Sure, go ahead and don't saved the generated code, we can always regenerate it". As someone who spent some time in frontend development, we've been doing it like that for a long time with (MB+) generated code, keeping it in scm just isn't feasible long-term. But given this…
Reproducible builds with deterministic stacks and local compilers are far from solved. Throwing in LLM randomness just makes for a spicier environment to not commit the generated code.
Earlier quoted context omitted.
I feel like using a compiler is in a sense a code generator where you don't commit the actual output
> I feel like using a compiler is in a sense a code generator where you don't commit the actual output Compilers are deterministic. Given the same input you always get the same output so there's no reason to store the output. If you don't get the same output we call it a compiler bug! LLMs do not work this way. (Aside: Am I the only one who feels that the entire AI industry is predicated on replacing only development…
Although I don’t really understand why you’d only want to store prompts…
What if that model is no longer available?