You don't call it fraud if your TV ad is aired and I miss it because I'm making a cup of tea. No matter how many viewers you were told the show has.
It's not fraud if I don't notice your roadside billboard because I'm focusing on driving. No matter how many cars you were told drive by.
Advertising has always been a crapshoot with a lot of supposed "TV viewers, magazine readers, eyeballs, etc.", but those figures have always had an assumption of "nobody knows which advert works". That's why an advert's success is generally measured solely by the increase in sales after it has been displayed (ignoring brand ads for the mo, although arguably they're the same).
It's the same when I've placed internet ads with Google myself. I place the ad, if I get more profit than the ad costs, it's a success. The % of ad clicks that are actually real does not matter. If the ad costs more than I make back, I stop the campaign. As Google run an auction system, if enough people stop bidding, then the ad price will go down and my campaign might become viable again. Either way, I'm not being defrauded anymore than I am by people making cups of tea during an ad break...
I think the only reason why people like to call it fraud is because the internet has given advertisers for the very first time, some hard(-ish) data on how many users are being influenced by their ads. In reality, it's just the old crapshoot, but with a little more accuracy than before.