No.
"Or should we make this (perhaps justifiably, after their misleading self-presentation) the first time we let someone go for performance reasons?"
You need to let them go. Whether you call it performance reason or not is your call. You made a mistake hiring the wrong person. It is your fault. But you need to correct it by letting them go.
Doesn't matter whether you are a small team or large. Wrong hire is a wrong hire and things won't magically change just because you wish to. Fix the problem now and go back to hiring mode. Thank me later. Been there many times.