Software or otherwise, the best way to find a good contractor is to get referrals from people who had a positive experience for similar work. That can be somewhere between hard and impossible though.
Failing that, and if you need to go out and just evaluate contractors you find through other means, the most qualified person for evaluating a contractor is another experienced contractor who is not competing for the work.
Hire somebody experienced, for a nominal but fair fee, to help you review your project plan and advise you through the sourcing process for its fulfillment. This is a different role than a recruiter, and in fact, be wary of anyone who wants to simply hand you off to someone they know without assessing other options. You want a personal advocate, not some contractor's sales agent.
Then, consider retaining them after the contract begins to double check the work as it proceeds, and to bring issues to your attention if things smell fishy.
Basically, if you are asking this question on HN and don't have access to a good referral, you're about to throw a huge bunch of money at a stranger that you don't know how to evaluate. Written guidelines won't be enough help. Because of your own inexperience, you're almost certain to either trust them too much, or distrust them too much, and either one is going to strain your project and cost you money and time.
Having an experienced advocate on your side, who has a different incentive structure than your contractor, can make all the difference. And unlike the implementing contractor that you'll eventually hire, you can successfully evaluate the advisor yourself. Their role is about communicating things to you, and advocating for you, and these are actions that you are totally qualified to judge. Plus, it's a smaller and more ad hoc role that doesn't have the sunk cost issues that you'll face with your implementation contractor. If you find that they're not doing a good job being your advisor and advocate, you can drop them without sending the project into crisis.
Besides myself, I don't know a lot of people who advertise this role directly, but I think many independent contractors would take it on if you pitched it to them. I think many working developers and development managers would too, as a side gig, although they may not know the norms and pitfalls of contracting quite as well. They'll at least be able to vet your project plan, review candidates for technical merit, and perform code reviews. Even just those tasks can make all the difference for you.
So long story short: before you hire a contractor, go poke around for a paid advisor who you think could help you hire one better. It's really worth it.