Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The short leash AI coding method for beating Fable

blog.okturtles.org

Post reply on HN
Print view View original

211–220 of 268 posts

Re: The short leash AI coding method for beating Fable

#211

Post by datsci_est_2015 »

andai wrote: post #179
chewbacha wrote: post #94

I did this for two weeks on a side project and still ended up in a situation where I did not have a mental model of the codebase. There’s no way build that model without building it yourself. I’m more convinced then ever of this.

Unfortunately I ran into the same issue even before AI. The forgetting curve means that my mental model lasts not much longer than the initial building period. As for how to rebuild it, I haven't figured that part out yet.

Sounds like you wrote very poor quality (edit: or trivial) code, or you’re exaggerating a bit for effect.

I too forget the details of most of the code I write, but the most important 10-20% of the code that I write encodes my mental model of the problem I’m trying to solve. Sometimes it’s a class representation of a digital or physical entity. Sometimes it’s a job with tasks that map to subproblems. Those abstractions almost immediately launch me into the mindset of my former self, even years (or a decade!) after the fact.

AI-generated code does not tend to create those kinds of abstractions in my experience. It will likely, with encouragement, solve the problem you’re asking it to - but it won’t magically cause you to understand how to solve the problem. You must take the initiative to understand it yourself. You are the camel that the AI has taken to water, and it can’t force you to drink it.

Re: The short leash AI coding method for beating Fable

#212

Post by naasking »

Jensson wrote: post #173
youdont wrote: post #153

Earlier quoted context omitted.

The human will quite convincingly be able to construct a post-hoc reasoning on an action that may or may not be related at all to what was actually going through their head or the actual instinctual reasons that led to a decision.

Humans can accurately retell what their consciousness was doing, but they have no clue why their unconsciousness responded as it did. LLM is just that unconsciousness part that humans have to post hoc explain like that, and lacks the conscious part that we humans actually can inspect in ourselves. If the AI had some introspection part where it actually tracks its reasoning maybe it would be closer to conscious humans…

> Humans can accurately retell what their consciousness was doing

Can they? How could we possibly know this is the case? People could simply post-hoc rationalize this to justify whatever decision they made.

Re: The short leash AI coding method for beating Fable

#213

Post by atq2119 »

cpursley wrote: post #196
dolebirchwood wrote: post #130

Earlier quoted context omitted.

> he :/

[flagged]

We can point out mistakes that feel rather grating without assuming intent behind them.

I agree that their use of "he" is likely because they're not a native speaker, especially because they're arguing against the capabilities of LLMs.

That doesn't make it inherently wrong to point out the mistake when it's so intertwined with the deeper discussion here, especially given the fact that some (hopefully few) people do build relationships with LLMs.

Re: The short leash AI coding method for beating Fable

#214

Post by dofm »

semiquaver wrote: post #29
moezd wrote: post #18

LLMs are still next token predictors, just because you can give it more vague instructions and it still finds the right steps to follow, it doesn't mean it's intelligent. It means you're speaking the same language as the harness they trained your model on. And that has a limit. If you are stuck at PoC level or simple apps, you have no idea how limited the current models still are. There you really need to break tasks…

Yeah, and you’re just a next-word-sayer.

I mean, conversationally, of course we work a little more like that (I tend to think in whole sentence blocks before I say them but I suppose they assemble themselves largely word-by-word, or word-by-word with a bit of editing).

But right now I am trying to design something -— a physical mechanism with a particular enclosure — that I cannot clearly describe (this makes it hard to research). I designed a previous version without even knowing the words that do, in fact, describe that.

I have a theory about it, animated in my mind, that I can only test by making it.

If I want you to know about it, I can either show you it or work out words to describe it, which will be inadequate to describing it.

The idea for it came from seeing things nobody has ever put into words for me.

"Next-word sayer" doesn't describe any of this process, does it?

(This is also why text-to-CAD is a bullshit idea)

Re: The short leash AI coding method for beating Fable

#215

Post by deadbabe »

From these comments I find it funny how mad some people get when someone finds success working without the latest “state of the art” AI slop methods. Some people really seem to have vested interest in pushing the AI coding supremacy. Probably people from Anthropic in here.

Re: The short leash AI coding method for beating Fable

#216

Post by NitpickLawyer »

atq2119 wrote: post #209
NitpickLawyer wrote: post #172

Earlier quoted context omitted.

You don't get access to the thinking traces. Might work with local models tho, but the current meta isn't particularly suited for this either, as it's a big blob of rambling surfaced by RL, with the "only" objective being that the thinking blob somehow leads to a better final answer. Something more detailed, using templates akin to oAI's harmony could work, provided there's also a step that teaches the models to refl…

That's true, but it does mean that the LLM itself actually does have access to those thinking traces and could therefore, at least in principle, answer what it was thinking. They're probably not trained to do that, though.

It depends. Up until recently the models were trained only to "think" on the last user message. So you'd send the message1, got back reply1 w/ think1 but you'd make the next iteration m1 - r1 - m2, and would get back reply2 w/ think2. You would not add the thinking1. That's how the models were trained, and that's how you were supposed to construct the conversation.

Now recently some things have changed, and you can add the thinking part (you get that encrypted from the closed API labs). But the model needs to have been trained for this to work. And doing it this way you'll burn through tokens faster, as the thinking parts are usually rather long.

Re: The short leash AI coding method for beating Fable

#217

Post by discreteevent »

andai wrote: post #179
chewbacha wrote: post #94

I did this for two weeks on a side project and still ended up in a situation where I did not have a mental model of the codebase. There’s no way build that model without building it yourself. I’m more convinced then ever of this.

Unfortunately I ran into the same issue even before AI. The forgetting curve means that my mental model lasts not much longer than the initial building period. As for how to rebuild it, I haven't figured that part out yet.

> As for how to rebuild it, I haven't figured that part out yet.

Just do some work with the code. If I go back and try to add a feature or fix some bugs on code that I have not worked with for a long time I find it much quicker to build up a mental model of it than code which I have never worked on previously.

Re: The short leash AI coding method for beating Fable

#218

Post by overgard »

doctoboggan wrote: post #122
sixtram wrote: post #106

Earlier quoted context omitted.

You say you can have increasingly nuanced discussions with stronger models. What I say is, when I asked Claude why he applied a certain change I didn't understand, and boy, it was a small change, he said he "reasoned from first principles" based on the code paths. But it didn't work, and when I asked, "Okay, describe the steps of your reasoning from first principles," it literally answered that it had just made it up…

You can never ask why a model did a certain thing, or what it was "thinking" when it said something - just like you can't ask a human which neurons were firing when they had a certain thought. The information just isn't available at that level. You absolutely can have deep nuanced discussions with LLMs however, you just need to better understand their strengths and weaknesses.

You certainly can ask it what it was thinking, the problem is just that it's more likely to make up a plausible sounding fabrication than to say "I don't know" or "my reasoning is hidden for business reasons" (frontier models hide a lot of their chain of thought). Which is the fundamental problem with LLMs though, if the data doesn't exist or it's sparse they make things up.

Re: The short leash AI coding method for beating Fable

#219

Post by solomatov »

I tried a similar approach before, but it didn't work for me. I didn't get a lot of speedup if any from it. IMO, to get productivity you need some kind of YOLO mode (in a sandbox).

IMO, the goal should be to outsource as much work to the model, as possible, while minimizing effort required to understand and review what is did. For example: ask the model to find out why a bug happens, figure out proof of concept for thing X, incrementally optimize something, do a well specified refactoring with some guide, and similar things.

IMO, what people say about creating loops is a very similar thing. You maximize the work done by the model, while minimizing the amount you need to do to control it.

Re: The short leash AI coding method for beating Fable

#220

Post by seunosewa »

arcanemachiner wrote: post #189
seunosewa wrote: post #186

Earlier quoted context omitted.

You can just ask the model to explain the code to you.

Yes, and you will still not understand as well as if you had built it.

We often don't understand the code we wrote 6 months ago.
Post reply on HN