Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The short leash AI coding method for beating Fable

blog.okturtles.org

Post reply on HN
Print view View original

241–250 of 268 posts

Re: The short leash AI coding method for beating Fable

#241

Post by andai »

discreteevent wrote: post #217
andai wrote: post #179

Earlier quoted context omitted.

Unfortunately I ran into the same issue even before AI. The forgetting curve means that my mental model lasts not much longer than the initial building period. As for how to rebuild it, I haven't figured that part out yet.

> As for how to rebuild it, I haven't figured that part out yet. Just do some work with the code. If I go back and try to add a feature or fix some bugs on code that I have not worked with for a long time I find it much quicker to build up a mental model of it than code which I have never worked on previously.

Yeah. Ebbinghaus found this in his work on memory in the 19th century. Even after something has been forgotten, re-learning it goes more quickly, as a function of how many times it has been learned already.

I'm developing a new memory system that functions as an L1 cache for the human mind, taking the opposite approach of Anki and showing you things you want "top of mind" as often as possible. (As opposed to as rarely as possible, which is the standard approach in the memory space these days!)

Re: The short leash AI coding method for beating Fable

#243

Post by datsci_est_2015 »

andai wrote: post #240
datsci_est_2015 wrote: post #211

Earlier quoted context omitted.

Sounds like you wrote very poor quality (edit: or trivial) code, or you’re exaggerating a bit for effect. I too forget the details of most of the code I write, but the most important 10-20% of the code that I write encodes my mental model of the problem I’m trying to solve. Sometimes it’s a class representation of a digital or physical entity. Sometimes it’s a job with tasks that map to subproblems. Those abstraction…

So the way I write code is that, my understanding is local. Okay, we need a function that does this (high level). It'll call these functions to do that. And then I just continue until there's nothing left to write and the thing works (after a few rounds of debugging). I understand each piece and what it talks to. But I can't hold them all in my mind at once, because there's too many pieces. (I think chunking helps he…

That’s fair I guess. I’m pretty consistently surprised by the wide variety of tasks that everyone under the “programming” moniker tackles. I consider myself a programmer by trade, even though I’m not a SWE. Your first paragraph couldn’t really be further from my personal experience. I haven’t thought in terms of functions in years, mostly “jobs”, “tasks”, “workflows”, “data flows”, “modeling”, “labeling”, etc.

Some people really do have jobs that I wouldn’t be surprised that LLMs will nearly completely automate away. And those people will be forced to move “up the stack” in terms of abstraction… but that’s already where I’m at. And LLMs are helpful, but I don’t feel threatened by them at all. If they take my job, I think computers will be declared obsolete. No more keyboards and mice.

Re: The short leash AI coding method for beating Fable

#244

Post by econ »

It's really an extension of the abstraction debate.

X86 was designed for performance. The language really hates humans compared to machine languages that came before. I thought it was a truly stupid idea at the time but had to change my mind eventually.

Then we glue on many layers of abstraction and made everything as convenient for the programmer as possible. Performance became unimportant!

It imho begs to question why we are even using x86 or risk or even FORTH if performance doesn't matter. Make something luxurious that doesn't need to be compiled? Perhaps plug and play coprocessors named after libraries.

But if we aren't going to look at the code anymore we might as well write the application in English and give the LLM some cache. Go full prayer driven development.

Re: The short leash AI coding method for beating Fable

#245

Post by solenoid0937 »

well_ackshually wrote: post #226
solenoid0937 wrote: post #111

Earlier quoted context omitted.

Posts like this are meaningless without more context - the model you're using, the harness, the initial prompt and context. Fable is better than most staff engineers at my FAANG.

Fable will definitely be the one on call when it inevitably breaks down from the pile of shit slop it wrote at 5AM, don't worry <3

We already use AI for oncall and it works better than our humans most of the time.

Re: The short leash AI coding method for beating Fable

#246

Post by lambdaone »

Jensson wrote: post #173
youdont wrote: post #153

Earlier quoted context omitted.

The human will quite convincingly be able to construct a post-hoc reasoning on an action that may or may not be related at all to what was actually going through their head or the actual instinctual reasons that led to a decision.

Humans can accurately retell what their consciousness was doing, but they have no clue why their unconsciousness responded as it did. LLM is just that unconsciousness part that humans have to post hoc explain like that, and lacks the conscious part that we humans actually can inspect in ourselves. If the AI had some introspection part where it actually tracks its reasoning maybe it would be closer to conscious humans…

I think a better way of putting this is that humans think they can accurately re-tell what their consciousness was doing. Whether they actually can, or even if consciousness exists at all as a thing outside the perception of consciousness is a philosophical question currently beyond answering.

Re: The short leash AI coding method for beating Fable

#247

Post by lambdaone »

loose-cannon wrote: post #234
doctoboggan wrote: post #122

Earlier quoted context omitted.

You can never ask why a model did a certain thing, or what it was "thinking" when it said something - just like you can't ask a human which neurons were firing when they had a certain thought. The information just isn't available at that level. You absolutely can have deep nuanced discussions with LLMs however, you just need to better understand their strengths and weaknesses.

Dude, these two things are not at all analogous: 1. Asking a model why it did a certain thing, and 2. Expecting a human to say which neuron fired in their response.

Even asking a human being why they did a certain thing is questionable. The research on choice blindness seems like a pretty definitive debunking of post-hoc rationalization:

https://en.wikipedia.org/wiki/Introspection_illusion#Choice_...

Re: The short leash AI coding method for beating Fable

#248

Post by FeepingCreature »

Dilettante_ wrote: post #222
FeepingCreature wrote: post #200

Earlier quoted context omitted.

Sufficiently constrained post-hoc justifications are indistinguishable from explanations. Consciousness tries to make things up, it learns that people notice this, it then begins trying to construct justifications that won't be predictably called out as false. Eventually it learns how its unconscious operates, and how to interrogate it, and its post-hoc justifications, at least in the common cases, become reliable.

> Consciousness tries to make things up, it learns that people notice this, it then begins trying to construct justifications that won't be predictably called out as false. There's a logical "skip" between that and > Eventually it learns how its unconscious operates, and how to interrogate it, and its post-hoc justifications, at least in the common cases, become reliable. The brain constructs a narrative that won't b…

Yep. It converges on truth unless there's a strong reward for lies because truth is easy. It's a neural network. It just reads off/probes the internal state because that's the cheapest way to model the unconscious. The justification won't necessarily be true, mind, in terms of the labels it puts, but it should mostly be true structurally- behaviorally predictive in the ordinary domain.

(Even if you are incentivized to lie and flatter yourself, it is still helpful to have access to the true signal internally, because that way you can know how to structure your lie to best avoid detection.)

Re: The short leash AI coding method for beating Fable

#249

Post by infinite_spin »

delusional wrote: post #225
infinite_spin wrote: post #164

Earlier quoted context omitted.

You're moving the goalpost. If the definition of intelligence is based on ability to "go build a church", then we've ruled out the vast majority of the animal kingdom from being labeled "intelligent". If you cannot be consistent in your definition of "intelligence", then you cannot have a reliable litmus test for it.

I wasn't trying to make a reliable litmus test for it. Either way, if you consider animals, LLMs are even more poorly positioned. They can do exactly none of the things my cat can do. An LLM can string together words, but if my cat is intelligent, it's clear that stringing together words is not synonymous with intelligence, since my cat can't do that.

Animals do in fact "string words together", e.g. parrots. You're also misidentifying what "language" is. Language in this context is not just the ability to string word together. Consider a musician, when they learn to play an instrument, they are learning the language of that instrument. Notes are tokens, ensembles are sentences and paragraphs. I'm afraid you're experiencing conformational bias, because every piece of evidence presented to you has been dismissed with things like "stringing together words is not synonymous with intelligence, since my cat can't do that".

Re: The short leash AI coding method for beating Fable

#250

Post by loose-cannon »

lambdaone wrote: post #247
loose-cannon wrote: post #234

Earlier quoted context omitted.

Dude, these two things are not at all analogous: 1. Asking a model why it did a certain thing, and 2. Expecting a human to say which neuron fired in their response.

Even asking a human being why they did a certain thing is questionable. The research on choice blindness seems like a pretty definitive debunking of post-hoc rationalization: https://en.wikipedia.org/wiki/Introspection_illusion#Choice_...

I'm not sure what point you're trying to make. In science and engineering, being able to provide justification is a core skill. The comparison we should be making is against the human practitioners who are trained in their fields. There will always be a distribution of ability. Saying that there's evidence that people are capable of providing post-hoc rationalization doesn't say anything about the ability of experts to produce well thought out responses (in their respective fields) that don't immediately fall apart under scrutiny.
Post reply on HN