Arguably I shouldn't even answer since I have no interest in a social app and would not try this no matter what it was, but there are some more specific issues related to the use of phone contacts.
Understand that, to anyone who owns a home or is registered to vote, a phone is effectively useless. Very nearly every call I ever receive is a refi offer or vote canvassing or someone who wants to buy my house. The purpose of a contacts list is a way to allow me to know I'm receiving a call from a source I actually want, or need, to answer. But mostly this means businesses. The numbers are associated with an office, not a person.
And, as another commenter noted, the numbers that are associated with persons are effectively anyone I ever had a relationship with and never deleted the number. That includes women I went on one date with 20 years ago, people who are now dead, neighbors who haven't been neighbors in decades. I don't have their consent to share that information with you.
I understand it's a way to bootstrap a social graph without being able to export a friend's list from some other social app, but I would urge you to reconsider. For all of the other reasons I will never again use Facebook, at least one thing I think they got right was just showing you the friends list of existing friends who had opted into sharing that information with you. That was probably the best use I ever got out of it, since it allowed you to discover people you used to know and might want to reconnect with, but had never gotten or lost their contact info. It's much slower to require users to add friends one by one, but at least this requires a rough form of symmetrical consent.
Phone numbers are not private. I have never changed mine. Effectively, everyone I have ever called and told my name to may have me as a contact, and any one of them could have listed the number somewhere else. That is obviously how I get on all of these constant contact or whatever they are call sheets. Someone scraped it from public property records and now I'm a contact for thousands of people I don't know. I would not ever want to allow those people to auto-add me to a social graph. It's just inviting yet another avenue for spam. Even if you don't directly inject ads into your service, there is nothing stopping your users from doing it. You're inviting companies like LuLaRoe to leech off of you if you ever grow successful enough.