1–10 of 20 posts
How do your abilities compare to the newest models?
edit: Please specify which model and version you are talking about.
They're still pretty dreadful. They're better than I was at 21, so I'd say they're good for graduate level, but nothing beyond that.
So this AI Agent... It is much faster at doing code when given specific instructions. But it keeps loosing context on architecture, and i cant really let it build complex things with interdependencies that build on each other. At times it feels like doing pair programming with a guy who is so crazy fast that im left behind with my head spinning, wondering how we just jumped from a hello world to a working thing that would have taken me ten iterations. And i get a bad feel when i then wonder how is this app doing what it does? because my agent cant explain it, and i would be stupid to believe what it hallucinated because it sounds really solid until you scratch the construction.
At the beginning i was almost euphoric about my new friend, now im sometimes disappointed, sometimes confused, but i am learning to give better, more concise instructions, to do smaller development jumps. It is tempting to set a long haul goal and let it do. But, i think for now, even if it is much faster at the small things, it would be also faster to build a catastrophic spaghetti code nightmare if not used with great care.
I don't use ChatGPT, but i've been using an agent with Claude Sonnet 4. My answer may not be useful to you, but i'll talk about my experience with that and hope it may help you. So this AI Agent... It is much faster at doing code when given specific instructions. But it keeps loosing context on architecture, and i cant really let it build complex things with interdependencies that build on each other. At times it fee…
Are you using Sonnet 4.6?
> So this AI Agent... It is much faster at doing code when given specific instructions. But it keeps loosing context on architecture, and i cant really let it build complex things with interdependencies that build on each other.
I've only built small things (Is it better than you at building small self-contained things?
> And i get a bad feel when i then wonder how is this app doing what it does? because my agent cant explain it, and i would be stupid to believe what it hallucinated because it sounds really solid until you scratch the construction.
Do you ask it to generate test suites for the things that it builds?
> it would be also faster to build a catastrophic spaghetti code nightmare if not used with great care.
noted
They're still pretty dreadful. They're better than I was at 21, so I'd say they're good for graduate level, but nothing beyond that.
Which models + versions are you using? Can you give a specific problem that you found them to be bad at?
It kept producing code that looked to eye that it might work, but each time I ran it it would just throw schoolboy exceptions. I got tired of telling it to correct the things it kept forgetting to check for (nulls, path starts, empty lists), and just coded it from scratch myself.
I find ChatGpt is like pair-programming with a junior, except I'm not getting paid to coach them like I would if it were an actual graduate hire.
They're still pretty dreadful. They're better than I was at 21, so I'd say they're good for graduate level, but nothing beyond that.
Which models + versions are you using? Can you give a specific problem that you found them to be bad at?
Given a long haul goal with instructions and everything they will reinvent the wheel four times and one of those you will get a square. Reminds me of that monkey paw wish thing. You look at your finished app. Looks beautiful, but its inner workings are a ball of confusion.
It is better at syntax and boilerplate. It writes cleaner code than I would have. But it is absolute shit at actually designing systems, in particular if you are integrating multiple platforms and stacks.
Is it bad at designing systems that don't have a bunch of integrations?