[dead]
Working With AI: A concrete example
31–40 of 78 posts
Re: Working With AI: A concrete example
#32[flagged]
Re: Working With AI: A concrete example
#33Carson’s experience matches mine: AI is good at analysis and boilerplate, but not good at the kind of critical thinking necessary for good designs. If it were human, I would say that it jumps to solutions to quickly, rather than stepping back to consider the big picture and how everything should fit together to make a cohesive whole. It’s not human, of course, and I think this problem actually relates to the fact tha…
Have to disagree with this as it's excellent at helping you wide and broad before converging. I suggest trying OpenSpec and use /ospx:explore to state your problem and go from there.
These takes Arne necessarily incompatible. It can be a great tool for helping you do this kind of big picture design, but still need you as a guide and taste-maker to get to a good end result.
Re: Working With AI: A concrete example
#34Carson’s experience matches mine: AI is good at analysis and boilerplate, but not good at the kind of critical thinking necessary for good designs. If it were human, I would say that it jumps to solutions to quickly, rather than stepping back to consider the big picture and how everything should fit together to make a cohesive whole. It’s not human, of course, and I think this problem actually relates to the fact tha…
One partial mitigation is to ask it to use plan mode -- and then very carefully review the plan before allowing it to execute.
My experience with AI plans is that they’re a wall of text that’s very hard to extract meaning from. Combined with it not doing a good job to begin with, I don’t think plan+revise is a great use of time.
Re: Working With AI: A concrete example
#35It's a good write up, but it's lacking some details, the most important one is: which Claude model was used? The second issue is: what was tooling and the prompt approach? (To be clear, I have no problem with the premise of the write up. But without some details like this, it's sort of like saying "I had a bad board on my deck, and my tape measure wasn't able to help me remove the nails. What a bad tape measure."
It's more like asking what editor and keyboard layout they use. Highly relevant to the user but you should simply assume someone describing work is using a setup for it they find productive. If you decide to dismiss their output it wouldn't be over these details.
Re: Working With AI: A concrete example
#36Earlier quoted context omitted.
Exactly, LLM is good at "code inpainting" : define clear structures and goals, and it will fill the boilerplate. But it doesn't work for reasoning and abstraction, so it fails to synthesise and propose novel views. But that's integral to the way it's designed and has been trained, to do a kind of "averaging" which limits it's capacity to explore novel designs
> But it doesn't work for reasoning and abstraction, so it fails to synthesise and propose novel views I disagree. Have a conversation with it about your problem and work through design decisions with it. When I do that, I find it gives me a lot of good ideas. Disclaimer: I'm not working on anything groundbreaking (like most people)
Yes, but "good ideas" compared to what? If you were aware of the better alternatives, you probably wouldn't be discussing those details with an LLM. You'd find that it just randomly gave you one. It might work, but you don't know how well until you're already entrenched.
Nobody knows everything, so of course LLMs can be useful sometimes. More useful than plain old search, books, or even discussion with real humans? Maybe.
Search can offer a much broader context than an LLM hyperfocused on just generating text. Books may lead you to realize you were asking the wrong questions. Discussions will provide an overall "vibe" of the topic.
These are not competing options. We can and should be using all of them when possible.
Re: Working With AI: A concrete example
#37Earlier quoted context omitted.
One partial mitigation is to ask it to use plan mode -- and then very carefully review the plan before allowing it to execute.
My experience with AI plans is that they’re a wall of text that’s very hard to extract meaning from. Combined with it not doing a good job to begin with, I don’t think plan+revise is a great use of time.
I feel the same way. Maybe it’s the ADHD, maybe I’m just dumb, but I cannot parse well the giant walls they tend to produce.
Re: Working With AI: A concrete example
#38Interesting read! Creating tests is highlighted as something Claude did well, but it strikes me that all the weaker rejected solutions could have been avoided if it were really good at designing intelligent tests for itself. For example, the first solution “was very specific to the reported bug and wouldn’t have fixed the general case” and the third suggestion “prevented the perfectly valid use of as conversion expre…
As humans we have a concept of viscosity. That resistance, like being in quicksand or a swamp, is how you “easily” identify a code smell, something that needs to be refactored, etc. Part of it is human laziness, part of it some concept of elegance, an itch of being not quite tidy as it can be, etc. LLM, being a tiresome little helper, will gladly output hundreds of lines, hacks, and what have you. I don’t think any a…
> “my shaman is a better shaman”
This made me chuckle. I will steal this from you.
Re: Working With AI: A concrete example
#39[dead]
Re: Working With AI: A concrete example
#40Earlier quoted context omitted.
My experience with AI plans is that they’re a wall of text that’s very hard to extract meaning from. Combined with it not doing a good job to begin with, I don’t think plan+revise is a great use of time.
I feel the same way. Maybe it’s the ADHD, maybe I’m just dumb, but I cannot parse well the giant walls they tend to produce.
It’s melting my brain to read them all day. Our merge request descriptions are a mile long and so dense with jargon that it’s very difficult to figure out the important part of the changes.
They turned the english language into enterprise java and my train of thought is now a series of NullPointerExceptions