It's relatively easy to change/fake the caller ID of phone calls so unfortunately this approach isn't really secure. That's why phone number verification usually places an outgoing call, to verify that you're actually able to receive calls on that number.
At the risk of sounding like I'm actually going to abuse this capability... if it's done "relatively easily", how is this done?
I believe the SMS verification is something that companies use to avoid liability alone, technically they can use any number as the caller ID if they choose to.