I did one of the phone polls about 15 years ago, and after about 30 minutes of questions I was really tired of the whole thing. But the last stroke is when instead of asking my opinions about issues/etc they started asking the personal questions (presumably to categorize me race, income/etc) I told them I was done answering questions and refused to answer anything that wasn't something you could conclude by looking at me on the street. Now I just say "no" and hang up.
And thats probably a big problem for them too, I had a few friends in HS who worked as pollsters at the local mall, they actually paid people in gift certificates and the like for the hour they took up. It was all really upfront, "Hi, would you like a $10 gift card for an hour of your time answering questions?". IIRC it was 100% issue questions too, the age/race/etc questions were all later filled out by the pollster from what they saw rather than being questions on the form.
So, yah, I'm guessing its a lot cheaper to run phone polls than pay people in the mall to give out gift cards.