I feel a really odd dissonance with all these accounts. I have access to top models via Cursor at work. Every time I think to myself, here’s a tedious but well defined task, with clear success criteria, where you can achieve a lot with persistent iterations. I’ll give it to Claude to do.
First, it takes me ages to describe the task properly. All these clear success criteria, well, instead of writing code from a clear spec in my head, I’m writing tons of prose, and trying to make it unambiguous. I’m programming in English++.
But then, one time in 5, it produces something that kinda works. Maybe not perfect but good enough. Two times out of 5 it kind of sort of looks alright, but actually ignores 80% of the spec, or pays it minimal lip service in the comments.
And sometimes, maybe a bit less than 2/5, it just completely diverges into total shit. Starts writing scripts that load the ast of the main script and pickle it then serialise to base64 for no good reason. Encode some stuff in strings then check ord(string[i]) repeatedly for string comparison. Eventually runs out of context and develops the LLM equivalent of severe dementia.
I truly cannot reconcile my experience with people who seem to say “hey computer write this” and it’s a good use of their time.