For example: when you speak, do you choose each and every word or do they more or less "come to you" and you choose/veto them based on some preference (or probability)? When you program, do you really think "hm, yes, should put a for here" or do you just look at the code you've written and your fingers start tapping away? It feels like you're choosing what to do since you're "overseeing" the whole process, but another part of you is also involved in producing the actual implementation. It feels very probabilistic -- the kinds of things you said yesterday are probably going to be similar to the ones you're saying today. Most people having a coding style.
When you have a cold for example you can find that you're perfectly able to think of what should be done, but have no ability to do so. The part of you that does coding is too "murky" and can't generate tokens anymore, and the overseeing part is too tired to motivate it back into action. A lot of programmers have ideas in the shower, in bed, after a run etc., further showing that there's a certain circuit we're unaware of that is doing some sort of work for us to be able to program.
In effect, what the AI missing is the "overseer" part, which perhaps is what we would identify as our own consciousness (the ability to think about the process of thinking itself). Given the incredible progress I think it's fair people have all kinds of SF ideas nowadays. I would have never thought something like chatGPT would be achieved in my lifetime, I've been playing with this thing for a while and it's amazing how well it can envision the physical world and interactions between objects it cannot possibly have examples for in the database.