OTOH prediction doesn't necessarily reflect causation either, but prediction is what JEPA is about, how our brain/intelligence works, and one of the great confirmations of LLMs is how powerful prediction errors are as a learning signal.
JEPA appears a step in the right direction of trying to build a brain rather than a language model - to use prediction the way the brain uses it to predict the future (not an historical frozen training set), and learn a real world model of how the world behaves. Any JEPA implementations I've read about use a Transformer as their predictive component since even prediction (and certainly not correlation) is not where JEPA is innovating - it is more about applying prediction to the right problem (assuming the goal is to implement animal/human intelligence) of predicting sensory inputs at the right level of representation.
A recent JEPA variant, Causal-JEPA, moves beyond just infilling to predict object state from object interactions (i.e. to learn causal predictive relationships).