(Not an expert manager; just some thoughts based on things I've seen.) I can't tell from the description whether the candidate was outright deceptive, like claiming experience they didn't have, or candidate was just good at the normal interviewing rituals, and the real problem was mess-up on company's end (i.e., interviewers said "mediocre" but HR somehow heard "hire at all costs"). In any case, talk with appropriate…
I'm not pro-HR by any means, nor have I had to hire/fire a bunch of people, but the minute you hint that an employee could be terminated, there's a chance that they could take retaliatory measures.
Further, OP stated, "Essentially we had deficiencies in our technical interview that allowed for custom-fit preparation..." So, it doesn't seem like the candidate had "good faith" intentions from the start.
If OP is in the US, most states have an at-will employment clause, so a quick break isn't illegal by any measure (as long as they aren't terminating the employee for things like sex, race, religion, etc.).