"the evilness comes from the fact that you have immense power"This is exactly right, IMO. If your product becomes successful, it will be incredibly tempting to tamper with the results to maximise your profits. You might think you are an honest person, and I'm sure you are, but what if one day you realise that just by making one little change, sending people to shop X instead of shop Y, you can make, say, $1000/day commission .. how many people wouldn't at least be tempted?
Being put in this kind of situation corrupts almost everyone. Because of this, I think people are very suspicious of affiliate arrangements.
I'd advise you to make it very clear that you're sending people to affiliate links - maybe even go over the top into radical transparency and explicitly give another link, the non-affiliate one. Put (Support this plugin!) and (Do not support this plugin.) after the links, give people the very clear option to choose one of the two. Make it clear that the non-affiliate link is exactly the same price as the affiliate one. If they feel like they are making the choice, many will no doubt opt to support you. But if they get the idea there is anything sneaky about it, they may get very hostile. I would really err on the side of transparency.
Anyway, just some ideas ..