I'm thinking a lot about the ARC-AGI ML benchmarks, especially the "shape" of the dataset and what that says about how it should be solved. I think there's good reasons to believe that deep learning - at least differentiable SGD backprop style - is a bad fit for this specific benchmark, due to the tasks being almost entirely discrete symmetries, and also having so little data to approximate the discrete symmetries with continuous ones (considering deep learning to be the learning of continuous symmetries). I think that a more explicit and discrete approach is the way to go, and it's possible to build something surprisingly general and not heuristic-based even without gradient descent, guided by minimum description length to search for both grid representations and solver functions. I'm looking for teammates for ARC-3 so hit me up if this sounds interesting, I'd love to chat!
I made a viewer on my website to build intuition for my preferred perception algorithm which is entropy filtering + correlation. Pretty neat to check out the heatmaps for random tasks, there is a lot of information inherent in the heatmap about the structure of the task: https://synapsomorphy.com/arc/