I took on three new hobbies a couple of years ago. All of them had seemed interesting to me for ages, but I'd never made the time to explore them. Once I actually started, it was exhilarating. Practice was long and slow, but never boring. I'd spend hours at it and not even notice the time go by as I worked at a particular technique (especially with classical guitar). Things only got irksome if I indulged in comparing my new skill to my skill level in programming, despairing at the difference, or if an injury prevented me from doing it for awhile. But after a couple of years of daily practice, I've finally reached a level of competence in two of them (there just wasn't enough time for three) where I can feel satisfaction in my output, and am even willing to show other people.
My early education in programming followed a similar pattern, actually. I'd actively pick the brains of any teacher in high school who gave off even a hint of understanding programming or anything related. Once I'd learned everything about digital logic and did all I could on the broken Heathkit boards, my electronics teacher bought a computer for the electronics shop and let me take over his office just to get me to stop bothering him. I also resurrected a 2400 baud modem and hooked it into his local phone line. Actually, I didn't tell him about that, or the fact that I was hacking into the local university to have a poke around gopher space :P. I got some Motorola manuals, and wrote a book on assembly language programming, which I handed in at the end of the year instead of doing the regular assignments & exams.
There are many things to be passionate about. It's just a matter of identifying which ones resonate with you, and making the time.