Well, it's interesting. I've been using LLM's for coding for over two years. My findings are somewhat consistent: Feed it jobs in small chunks and it can be a great auto-complete, beyond that and it turns into a mess very quickly.
I'm two weeks into a project that has been interesting and also has confirmed the above yet again. I am porting a Windows application written in Python to C# using Avalonia UI.
I have never used C# or Avalonia UI. I've written Windows applications in other languages, never C#. I don't know the language, libraries, etc.
At first I told Codex: Here's the source code, port it. I just had to run that test.
Well, it didn't end well. I'll describe it as a frustrating set of prompts that seemed to result in the implementation going in circles with constant problems being introduced, breaking-fixing-breaking, etc.
I then started again with a clean repository and played the role of the architect with full documentation in the form of code. File-by-file, I had it port modules to effectively develop an operational foundation for the classes, methods, properties, abstractions, hardware interfaces, etc. in the original Python program. That went well, was very fast and a good experience. I am running Codex in JetBrains Rider and the integration is excellent. The native OpenAI Codex application is an absolute dog...it pegs all my cores at 100% while doing nothing.
Once all the underlying infrastructure was ported and, to the extent possible, individually tested, I threw UI integration at it. This happened quickly and OK from it's-ugly-but-I-can-use-it perspective. Codex seems to be way out of its element when it comes to UI/UX understanding. Funny examples like placing a button on top of an image with the image covering the button because it had a higher z order. Or finally placing the button at the correct z order but making it transparent on-hover. Funny stuff when you are moving slowly and you see it happen. It just proves that there is no understanding whatsoever.
With all of that and the experimentation, I'll estimate that a six month project will be cut down to three to four weeks. Another month and it will probably be a much better program with new features and more advanced capabilities.
And I have not touched a single line of code. Developing solid prompts is the absolute key, something that you can only really learn by doing and through lots of experimentation.
Yes, of course, I fed it working code. I think it could be very different if I were to start something from a blank slate.