When we think of brain and size, we naturally think of the connection to the intellect. But intellectual activities require patient and slow application of the brain, using external tools like making notes, drawing diagrams and checking one's work for mistakes.
Intellectual pursuits are not actually what the brain is good at. If we make an analogy to computing, it's almost like the intellect is a software layer that is bootstrapped on top of the brain in spite of its shortcomings, using some slow, inefficient scripting language. It's not native to the brain.
An example of what the brain is natively good at is real-time visual processing and control, like navigating in a complex environment full of obstacles and dangers. You don't have to concentrate your intellect for minutes at a time to tell apart a shape from the background; you just point your eyes at it and it pops out at you instantly and effortlessly, thanks to a power in the brain that you can't even control consciously. (Try as you might, you can't not see the shape.)
This is probably why domestication shrinks brains. Perhaps you don't need that much real-time processing in the relatively stable, calm, boring domestic environment.