231–240 of 376 posts
Earlier quoted context omitted.
Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.
Humans have the ability to retrospect, push back on a faulty spec, push back on an unclarified spec, do experiments, make judgement calls and build tools and processes to account for their own foibles.
Earlier quoted context omitted.
> That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. LLMs can generate (relatively small amounts of) working code from relatively terse descriptions, but I don’t think they can do so _reliably_. They’re more reliable the shorter the code fragment and the more common the code, but they do break down for complex description…
Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.
> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…
“LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions” Only with well-known patterns that represent shared knowledge specified elsewhere. If the details they “fill in” each time differ in ways that change behavior, then the spec is deficient. If we “figure out” how to write such detailed specs in the future, as you suggest, then that becomes the “code”.
After awhile, I think we all get a sense of not only the amount of micro-decisions you have to make will building stuff (even when you're intimate with the domain), but also the amount of assumptions you'll need to make about things you either don't know yet or haven't fully fleshed out.
I'm painfully aware of the assumptions I'm making nowadays and that definitely changes the way I build things. And while I love these tools, their ability to not only make assumptions, but over-engineer those assumptions can have disastrous effects.
I had Claude build me a zip code heat map given a data source and it did it spectacularly. Same with a route planner. But asking it build out medical procedure documentation configurations based off of a general plan DID NOT work as well as I had expected it would.
Also, I asked Claude about what the cron expression I wrote would do, and it got it wrong (which is expected because Azure Web Jobs uses a non-standard form). But even after telling it that it was wrong, and giving it the documentation to rely on, it still doubled down on the wrong answer.
Earlier quoted context omitted.
Humans have the ability to retrospect, push back on a faulty spec, push back on an unclarified spec, do experiments, make judgement calls and build tools and processes to account for their own foibles.
which bits of this do you think llm based agents can't do?
The vibe coding maximalist position can be stated in information theory terms: That there exists a decoder that can decode the space of useful programs from a much smaller prompt space. The compression ratio is the vibe coding gain. I think that way of phrasing it makes it easier to think about boundaries of vibe coding. "A class that represents (A) concept, using the (B) data structure and (C) algorithms for methods…
But there is an entire cohort of people who can think about specifying systems but lack the training to sdo so so using the current methods and see a lower barrier to entry in the natural language.
That doesn't mean the LLM is going to think on your behalf (although there is also a little bit of that involved and that's where stuff gets confusing) but it surely provides a completely different interface for turning your ideas into working machinery
> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…
“LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions” Only with well-known patterns that represent shared knowledge specified elsewhere. If the details they “fill in” each time differ in ways that change behavior, then the spec is deficient. If we “figure out” how to write such detailed specs in the future, as you suggest, then that becomes the “code”.
Earlier quoted context omitted.
which bits of this do you think llm based agents can't do?
The main thing they cannot do is be held accountable for any decisions, which makes them not trustworthy.
Earlier quoted context omitted.
Humans have the ability to retrospect, push back on a faulty spec, push back on an unclarified spec, do experiments, make judgement calls and build tools and processes to account for their own foibles.
which bits of this do you think llm based agents can't do?
Earlier quoted context omitted.
Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.
Humans have the ability to retrospect, push back on a faulty spec, push back on an unclarified spec, do experiments, make judgement calls and build tools and processes to account for their own foibles.
The best LLMs can manage is "what's statistically-plausible behaviour for descriptions of humans in the corpus", which is not the same thing at all. Sometimes, I imagine, that might be more useful; but for programming (where, assuming you're not reinventing wheels or scrimping on your research, you're often encountering situations that nobody has encountered before), an alien mind's extrapolation of statistically-plausible human behaviour observations is not useful. (I'm using "alien mind" metaphorically, since LLMs do not appear particularly mind-like to me.)
Earlier quoted context omitted.
The main thing they cannot do is be held accountable for any decisions, which makes them not trustworthy.
This is not correct. They can say "sorry" which makes them as accountable as ordinary developer.