Live data from Hacker News

The short leash AI coding method for beating Fable

blog.okturtles.org

81–90 of 268 posts

Re: The short leash AI coding method for beating Fable

#81

Earlier quoted context omitted.

This is wrong. Human thinking and speech isn't autoregressive like LLM inference.

Do you not say your words one-at-a-time like everyone else? Otherwise I can’t see how my comment is “wrong”

> Do you not say your words one-at-a-time like everyone else

You're conflating being autoregressive with being sequential.

Re: The short leash AI coding method for beating Fable

#85

I feel like OP is still in the year 2025. > The AI will have gone off the rails multiple times and you will only notice it later when you actually try to use the software. Except that said AI can now themselves use your software and find and fix bugs themselves, not to mention drive new features. >Your agent might go “off the rails” and start doing something you don’t want it to do This happens but far less often tha…

> This happens but far less often than it used to, and the case for full autonomous agents is getting stronger, not weaker.

This is that I do not see. My journey, just couple weeks ago, Claude Code + Opus 4.8. The task was not too complicated, 4 new API endpoint plus events streamed from client by websocket.

1. Multiply iterations on API definitions, refine request/response models, database schema, whole flow. A lot of corrections, removing contradictions, manual changes in document. Opus went of rails all the time. 500+ lines final document

2. API Integration tests. Once again, back and forth. AI was unable to create tests directly from document, so 2 iterations: Create placeholders with Given-When-Than comments, review an correct by hand. Second iteration was to implement tests. A lot of mistakes corrected after review.

3. Implementation. CC got api document, working tests ( modifications blocked by hook ), 6+ "best practices" skills ( most promptly ignored ), "rubber duck" and "code simplifier" agents, pre cooked scipts to run tests, linter, and check for compilation errors. Plan + execution + review, multiply corrections on the way. Feature implemented, all tests passed.

4. Code review. At average, found one issue per 20 lines of code. Not count code style, things like: Use in memory semaphore in kubernetes service (deployment described in CLAUDE.md ), 8 database calls to update the same record during a single request. One column at a time! Read-modify-save without transaction. Mistakes in business logic, failure recovery, authorization.

The result: almost one workweek, $100+ in tokens, and one thought: did it worth the effort ? P.S. I have a team of 2 developers. Just got PR to review from one of them. 80% slop.

Re: The short leash AI coding method for beating Fable

#86
post #52

Earlier quoted context omitted.

Even if you could understand human cognition to the level required to say, confidently, that it’s done one word at a time, it’s likely not! Natural language is not a prerequisite for human intelligence, as evidenced by the fact that we went from primates to commenting on HN. Natural language is, however, a prerequisite for the existence of LLMs. It’s more similar to methods for storing and retrieving information, lik…

Deepseek zero didn’t mix up all languages in something very efficient?

Interesting thought but I assume a lot of samples in the training corpus are examples of translation between languages and the same text in different languages.

Re: The short leash AI coding method for beating Fable

#87

There really wasn't much substance to this article.

It’s just parroting the current trope. Last year it was, “AI is just a stochastic parrot.” This year it’s, “AI can write the code, but a human still has to review it!” (Using AI, of course.) Give it another year and the narrative will be: “Only AI is capable of reviewing code, and only AI can review the AI’s review. Humans just need to read the AI’s final opinion so they still have meaningful oversight.” The goalpost…

Why shouldn't the goalposts move? That it was possible to beat or tie a chess master, if you had enough computational power, was basically the content of a theorem of Zermelo over a hundred years ago. It differs not a whit from tic-tac-toe. Even Eliza was practically passing the Turing test, which seems comically silly now. There's just an incredible amount of computational power so all sorts of things are possible that were formerly unimaginable - like training LLMs on the whole corpus of extant human discourse.

Re: The short leash AI coding method for beating Fable

#88
post #66

Earlier quoted context omitted.

> I think we're mving towards humans no longer needing to understand a codebase, and letting AI drive it. Hard disagree. Even the best frontier models generate output that's not what I asked for. Sometimes I realize that I get lazy in my prompting and the lack of specificity winds up showing up in the output. Just the other day, a coworker built a huge feature using frontier models and it slipped an IDOR in. I just d…

I can't help but feel that this reads more as a reflection that you don't want to stop being a developer than it does that thing's aren't moving in the direction that the GP said it is.

Maybe, it seems like a bad idea for so many reasons though. Take away tactile code review, insert a layer of prompts and tooling between developers and the codebase, and you've created the conditions to let all kinds of nefarious things happen in a codebase. A disgruntled employee updates agent prompts instructing the code review bot to ignore data exfiltration vulnerabilities (because if we aren't reviewing code, we're probably not reviewing prompts either), ships a backdoor, and you better hope that your network monitoring catches it.

Re: The short leash AI coding method for beating Fable

#89

I feel like OP is still in the year 2025. > The AI will have gone off the rails multiple times and you will only notice it later when you actually try to use the software. Except that said AI can now themselves use your software and find and fix bugs themselves, not to mention drive new features. >Your agent might go “off the rails” and start doing something you don’t want it to do This happens but far less often tha…

> This happens but far less often than it used to, and the case for full autonomous agents is getting stronger, not weaker. This is that I do not see. My journey, just couple weeks ago, Claude Code + Opus 4.8. The task was not too complicated, 4 new API endpoint plus events streamed from client by websocket. 1. Multiply iterations on API definitions, refine request/response models, database schema, whole flow. A lot…

Same thing I'm seeing, all the "AI practitioners" at my company with their advanced workflows are just shipping mountains of slop, and end up either putting the actual work on the reviewers, or the poor soul that's on call when an incident occurs.

I feel like people that have built crazy AI workflows have developed a false sense of confidence that their guardrails are helping them ship clean/correct code with little review when it isn't the case at all. In reality, the models and harnesses are at a point where there's very little difference as long as your prompts are somewhat reasonable, and the quality of the code ultimately comes down to the level of care and effort the implementor puts into it.

I don't think the first people that are going to be replaced by AI are going to be the people who don't use it extensively. The first that will be replaced are going to be those that are using AI mindlessly, because at that point, what are you besides a very expensive human LLM interface? To be clear, I'm not "anti-AI", I use AI quite extensively (in a way that's similar to what's described in the article), I just think that it's being pushed in a completely unsustainable way and the industry is in a collective psychosis over it's capabilities.

Re: The short leash AI coding method for beating Fable

#90

I feel like OP is still in the year 2025. > The AI will have gone off the rails multiple times and you will only notice it later when you actually try to use the software. Except that said AI can now themselves use your software and find and fix bugs themselves, not to mention drive new features. >Your agent might go “off the rails” and start doing something you don’t want it to do This happens but far less often tha…

Whatever we're moving toward, I currently can't let any SOTA model + harness operate on more than ~10k changed SLOC at once, and even then only with very careful prompting I thoroughly understand, only on the simplest of problems, and only if I pause it at key points to correct some sort of nonsense thinking and put in a significant cleanup pass and am still willing to tolerate some bullshit. Tooling is impressive for sure, but it's not magic.
Post reply on HN