Loading over 30 Kb of minified/gzipped Javascript over a 3G connection on a mobile phone is very problematic, as the bandwidth is poor, the CPU capacity for decompressing and parsing is poorer than on the desktop, so the latency can many times be measured in seconds, therefore loading over 30 Kb of stuff that you don't need gets to be a tough pill to swallow. Also the argument of caching is often brought up, but in my tests this can be very non-deterministic, as mobile phones are pretty aggressive in their used storage. JQuery is supposed to be cached, yet in our A/B testing the load times and consequently the conversion rates improved greatly after we dropped JQuery.
Your comparison with "a single medium-resolution highly optimized image" doesn't hold for mobile phones. On mobile phones users expect latency when loading images, but not when loading the UI. Another problem is that JQuery is not the only commonly used library. You add Underscore, you add Angular, or whatever libraries du-jour we have and pretty soon we are talking about 100 Kb of stuff or even more that is rarely used as a whole.
Of course, my use-case is about having really simple interfaces augmented by Javascript and meant for mobile phones. Well, for complex interfaces I think bringing in JQuery as a dependency may be worth it, but then again, I then start thinking that JQuery is not enough. It's because I want more, I want a virtual DOM that's saner by design, I want to work with immutability, I want much better performance without unpredictable behavior (like what you get with Angular) and I want to only pay for the features I use. JQuery is incompatible with React and is incompatible with Google Closure in advanced mode.