A friend and I built a similar service back in 2006-2007. It was actually a rev share program based on affiliate aggregators like LinkShare and Commission Junction -- that's how we populated our product list. A quick look at some of the sample vendors mentioned and the affiliate programs they use, I suspect you are doing something similar.
Problems we ran into included: trying to normalize the product data from the vendors into a format usable to make actionable and accurate ads--the aggregators practiced very little data hygiene and did not enforce much consistency, oddly, some vendors couldn't even spell the names of their products correctly which made for some pretty crappy looking ads; at one point we had over 4 million products listed, primarily because we started importing Amazon products and vendors, so the product selection was difficult, some things have general appeal and would show up all the time and others wouldn't match much content at all; we had trouble signing up sites to display the ads, CPA was relatively new at the time for the scale we were going for, we might have been better off selling our tech to the top affiliate aggregators as a better, easier method for sites to integrate affiliate program membership; reporting was a real bitch because of the delay from click through to actual purchase to the payment showing up, which depending on the affiliate program could be upwards of 90 days.
In retrospect, we would have better off setting it up for passive income and not try to do active sales to integrator websites and instead shored up the pitch copy and made it largely self service for the integrators (it mostly was already, but I spent a lot of time tweaking the matching algorithm rather than on the sales/pitch side), and pitched it as an easier way to join affiliate networks. That would have set expectations a little better than calling it an "ad network" like AdSense, which we also considered our main competitor.