For the record what kind of chargeback are you initiating, and why does it have to go through visa rather than the bank who issued you the card? Unauthorized card-not-present transaction initiated by a third party? Some cbs are harder than others to get ruled in your favor, but the one where a criminal takes your card and uses it without your knowledge is by far the easiest one to get awarded. It involves one call to your bank and you get a new card, all fraudulent charges reversed.
If your bank doesn’t want to honor the request yes you’ll have to contact the payment network (visa/mastercard) and I’m sure there’s someone in this thread who has experienced that for an unauthorized transaction chargeback but it’s exceedingly rare.
Merchant error chargebacks , on the other hand… very different situation.