Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The short leash AI coding method for beating Fable

blog.okturtles.org

Post reply on HN
Print view View original

161–170 of 268 posts

Re: The short leash AI coding method for beating Fable

#161

Post by dncornholio »

chewbacha wrote: post #94

I did this for two weeks on a side project and still ended up in a situation where I did not have a mental model of the codebase. There’s no way build that model without building it yourself. I’m more convinced then ever of this.

That's why I like to build a complete feature and the infrastructure myself first, so the AI will have a picture of how the code should look and where it should live.

Or I use the short-leash method and I will instruct the AI build infrastructure first, without even talking about features yet.

Re: The short leash AI coding method for beating Fable

#162

Post by Shitty-kitty »

youdont wrote: post #153
Shitty-kitty wrote: post #152

Earlier quoted context omitted.

A human won't respond with "Neuron 10-100 of the frontal cortex" (jokes aside) with deceptively convincing confidence.

The human will quite convincingly be able to construct a post-hoc reasoning on an action that may or may not be related at all to what was actually going through their head or the actual instinctual reasons that led to a decision.

That's exactly what the LLM seems to have done as well. The problem is that we want and even expect the A.I to be truthful.

Re: The short leash AI coding method for beating Fable

#163

Post by dncornholio »

ACCount37 wrote: post #39
moezd wrote: post #18

LLMs are still next token predictors, just because you can give it more vague instructions and it still finds the right steps to follow, it doesn't mean it's intelligent. It means you're speaking the same language as the harness they trained your model on. And that has a limit. If you are stuck at PoC level or simple apps, you have no idea how limited the current models still are. There you really need to break tasks…

"Next token prediction" is an interface, not an algorithm. A process that "predicts next tokens" can be arbitrarily complex or simple, and arbitrarily capable or incapable of performing a given task. Saying that an LLM can or can't do something because it's a "token predictor" is a category error. The interface isn't a hard limit.

The word "prediction" still holds a lot of weight. LLM's only can predict what has been written. This is a hard limit.

Re: The short leash AI coding method for beating Fable

#164

Post by infinite_spin »

delusional wrote: post #128
infinite_spin wrote: post #54

Earlier quoted context omitted.

You're conflating a singular model with a much larger system, but I want to address some of your points anyway. > An LLM has a fixed number of ways it can express itself While deterministic, there is not a fixed number of ways it can express itself, given that we can use settings like temperature to inject randomness into the output. > An LLM does not persist in consciousness from one token to the next While a model…

> While deterministic, there is not a fixed number of ways it can express itself, given that we can use settings like temperature to inject randomness into the output. You're missing the point, which is that no matter the process involved. The LLM can only ever output one of the tokens in its token vector. It can't invent a new symbol or character. It can't leave and go build a church. It has to output a little piece…

You're moving the goalpost. If the definition of intelligence is based on ability to "go build a church", then we've ruled out the vast majority of the animal kingdom from being labeled "intelligent". If you cannot be consistent in your definition of "intelligence", then you cannot have a reliable litmus test for it.

Re: The short leash AI coding method for beating Fable

#165

Post by infinite_spin »

moezd wrote: post #112
infinite_spin wrote: post #35

Earlier quoted context omitted.

> it doesn't mean it's intelligent I'm not sure how you're defining "intelligent", but I'd like to know how it is able to exclude a language model, while still including humans, without simply defining it with an axiom that predefines LLMs as lacking intelligence.

Intelligent humans are capable of following diverse and intricate analogies and draw lessons from seemingly unrelated events. Try asking an LLM to summarize an article and use an imprecise way to state your view. Ask it to push back. You will be drawn into so many pedantic arguments that burn through your tokens within a few messages, you'd wonder if there's someone deliberately taking over the keyboard on their side…

Agentic systems use LLMs, and they are absolutely able to follow diverse and intricate analogies. I use them frequently to hunt down notoriously difficult to find memory leaks, in codebases too large for a human to read in a single sitting. They are able to not only follow those intricate paths, they're able to discover solutions and apply those solutions. I use these systems quite a bit, and it's nothing like you've described.

Re: The short leash AI coding method for beating Fable

#166

Post by RealityVoid »

sothatsit wrote: post #126
RealityVoid wrote: post #110

Earlier quoted context omitted.

I am a bit confused which part you disagree with specifically. Reading AI responses and reviewing code seems to be what you propose as well. Your example with MLIP is something that would not be prevented by this approach, during the planing phase, it would surface. I guess the devil is in the details and the way you prompt it for starting the task matters. But IMO you absolutely need to check the output, need to eng…

I disagree with keeping an eye on the model as it is working, approving every command, and denying and stopping the model when you think it has gone wrong. It is not that it is actively harmful to do this, but rather that it is a waste of time and you can avoid the need for it through better design discussions and review. Micro-managing and keeping the AI on a "short leash" also lends itself better to telling models…

I personally am somewhere between you and the author. I don't check _all_ the intermediary steps, but I do try to understand what it's doing [1] and follow the process. Mostly I let it do the changes itself without supervision at each step but when a coherent "chunk" of work is done, I go through it really thoroughly. In almost 90% of the cases after a chunk is done some adjustments are needed.

I find broad architectural design to be _better_ if you follow along in the process because you better understand the direction it's going earlier and you can shift the high level direction much earlier. Even if you check its steps, you can ask it for its take on high-level architectural aspects along the way, no problem. I think personal touch matters a lot though, because I naturally ask it and try to get the big picture image.

[1] I actually find it really instructive what tooling it uses to tackle a problem, I got to become a much better console user because of it

Re: The short leash AI coding method for beating Fable

#167

Post by embedding-shape »

ramon156 wrote: post #148

One problem I have with "how to do X with AI" is that every situation is different. For example, I'm bumping Symfony projects from 3.1 to 8.1. There's a clear path here - Follow the written up migration guides PER major version - test all routes, authorised, etc. You can even hand-curate these tests. some might return 200, some might return 302 - Maybe optionally start with writing a safety net so you do not need to…

> One problem I have with "how to do X with AI" is that every situation is different

It's less of a "problem" and more a "How to approach content on the internet". Everyone is writing things from one perspective (usually) while there is a wide-range of perspectives out there, and what works in one situation doesn't work in another. "software engineering" as a whole is basically figuring out what goes where, and when, then trying to ignore the rest.

Then lots of company blog posts wants to lead you to believe there are silver bullets, solutions that apply for every scenario and case out there, which usually isn't true.

So again, less of a "problem" and more of a "Some things work in some situation", like we've been dealing with forever in software engineering. It's not right, it's not wrong, just applied practically different in different situations, perfectly fine and normal.

Re: The short leash AI coding method for beating Fable

#168

Post by user43928 »

sixtram wrote: post #154
doctoboggan wrote: post #122

Earlier quoted context omitted.

You can never ask why a model did a certain thing, or what it was "thinking" when it said something - just like you can't ask a human which neurons were firing when they had a certain thought. The information just isn't available at that level. You absolutely can have deep nuanced discussions with LLMs however, you just need to better understand their strengths and weaknesses.

So you're saying I can absolutely have a deep, nuanced discussion with an LLM, as long as I don't ask how he arrived at his conclusions?

Have you not seen all the posts with claims that AI lies about its reasoning when asked to explain how it arrived at the output?

I would instead ask the model to explain how X works, whether it achieves Y, and why we cannot do Z instead.

That is how you have a discussion with the AI.

Re: The short leash AI coding method for beating Fable

#169

Post by sn0n »

youdont wrote: post #153
Shitty-kitty wrote: post #152

Earlier quoted context omitted.

A human won't respond with "Neuron 10-100 of the frontal cortex" (jokes aside) with deceptively convincing confidence.

The human will quite convincingly be able to construct a post-hoc reasoning on an action that may or may not be related at all to what was actually going through their head or the actual instinctual reasons that led to a decision.

Isn’t that part of what the think blocks are for? Yea, don’t inject them back into the context, but do log them for review of that train of thought… no?
Post reply on HN