Earlier quoted context omitted.
I remember reading an interview with a fireman who described a time when his buddy evacuated a team because he "felt" that a floor would collapse imminently. He couldn't articulate why but they trusted his gut and it did collapse. A lot of software engineering relies on that kind of intuition and on a good team you can integrate it and benefit from it and avoid all manner of floor collapses.
To play devil’s advocate, intuition is still a physical response to stimuli mixed with knowledge of past experience. Hypothetically it could be modeled- the problem here comes down to how to encode it.
I'd argue that transformers are a pretty good indication that intelligence isn't "encodable" in the way we think it means. Usually, most "model" vocabulary means that we can explain and constrain the "data" from the "rules". Except the mere "data" is trillions of interacting weights.
That may be encoding in a physical sense, but that still doesn't explain the intuition in any legible way to humans.
Cynically, we've been able to encode everything already by just saying everything's a transition in a huge lookup table. Not very informative though.