381–390 of 434 posts
Earlier quoted context omitted.
Or in a way, nothing at all.
On its own, yes, but not in context. I think one of the problems with Claude's stock output is it assumes the reader has a firm grip on the context of the output, which is often untrue. Stock output is exhausting to read because you have to unwind metaphors in an unstated context.
This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these mode…
Lately, I've been having less and less success using the models for work. Simple things like: 'write this in a separate file' (writes it in the same file) 'format this aiming for 5 LoC' (emits newline after every comma, resulting in 17 LoC) 'include 5 warmup steps before you measure runtime' (omits it completely and apologizes after I point it out) I wouldn't be as opposed to using the models if they weren't as unrel…
Earlier quoted context omitted.
My point is that I understand all of that but in my view that makes them fundamentaly flawed at a conceptual level for what people are saying they should be used for.
Why? I'm an "artisanal programmer" at heart, but for the past year I've been using these to great effect. A car that needs to be started with a hand-crank and occasionally stalls is still a car.
Earlier quoted context omitted.
Maybe I’m behind the curve here because I work in an SRE/DevOps context as of late - but LLMs routinely shit the bed and fail to solve basic issue for me when I try to use them (Thanks, Management) I don’t need a model to shit out a REST endpoint. I need it to figure out esoteric errors that take hours or days of debugging. They just don’t do well here. Of course, if a diligent engineer refined considerations from a…
Further evidence that there is some kind of weird parallel universe thing going on with LLMs. "Figuring out esoteric errors" is one of the things I would cite as a particular strength of agents. I am repeatedly amazed at their ability to root-cause weird behavior on my systems. Here is one example: https://xcancel.com/lukechampine/status/2047032091053859138
Earlier quoted context omitted.
I don’t understand why this one is all the way down here despite being the only reply that answers the questions asked. And the answers are right: We all do our work based on the context. We would process the same request differently under different circumstances. System prompts (and user-made AGENTS.md on top of that) are simply the very didactic and direct way to provide that context to LLMs. I guess it would be mo…
My guess is people are upset that this technology is getting pushed on them and through the industry with the promise (and shown side effect) of replacing them permanently. The failure modes of adopting this technology (early) then corroborates their anger (slop code, employees being lazy/deferring their AI bugs on them, unrealistic mania in the industry, the obvious grift, etc.). And this is all off the heel of larg…
Earlier quoted context omitted.
Why? I'm an "artisanal programmer" at heart, but for the past year I've been using these to great effect. A car that needs to be started with a hand-crank and occasionally stalls is still a car.
An artisan who uses a mass production factory to make their goods isn't really an artisan.
I do hand programming for myself now and use the right tools in my office, because I'm a responsible adult who understands that I am not paid to have fun or feel special and smart - it's to produce a product for my employer.
We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...
I am quite tired of this take, frankly. The implication is that if we continue iterating on prompt optimization, we're going to reinvent what, JavaScript? BASIC? Lisp? English is not a programming language. Yet English is sufficient to communicate requirements to the degree that we actually care about. A programmer's job is to translate English into lower-level machine language. Necessary to this process is "filling…
Earlier quoted context omitted.
I’m ashamed to admit that I once suddenly realized what I had done: I asked the LLM to rename a variable. It took it some minutes or so. With IDE by hand it would have been seconds.
I'm ashamed that I use a programming language where the IDE can't rename things, so I rely on AI to do it...
Earlier quoted context omitted.
One day it will.
Not with LLMs, it won't. (The probabilistic prediction property prevents proper persistent programmable predictability)