Earlier quoted context omitted.
If he simply doesn't want to talk about something, isn't he entitled to take the 5th and just refuse to answer? Saying he doesn't recall is a gambit because it's an obvious lie but I think he's hoping it's one we're so used to hearing that it sounds less incriminating than refusing to answer.
I don't believe you can selectively take the 5th. As in once you waive your right to the 5th and take the stand, you have to answer all of the opposing side's questions truthfully. (Not a lawyer.)
What if you don't know the answer, are only probabilistically sure of the answer, or the answer isn't a simple yes or no?
I see lots of courtroom videos and congress videos where the idiot keeps pressing for a "yes or no" but the reality of the universe is that lots of things are more nuisanced than just a yes or no.