Using a neural network for things that have clear cut rules is wrong. When you know the exact rules, implement them as such, instead of bruteforcing a guesstimation. This is also why I'm sceptical of the usr of GPT-3 for all sorts of purposes where accuracy is important. Think of the code generation case. Bugs may be very subtle and may go unnoticed.
Code generation only needs to generate code with n bugs where n is less than the number of bugs a human developer generates for it to have usefulness, and maybe some other factor of severity where they are generally less severe than human developers. I think it'll make neat autopilot functionality for developers but not replace the need to have someone look over and understand the code.
Only every time I read something similar, I think "surely no programmer could think this". Are you a programmer?