These days I contract so it never comes up - but I used to resist requests to help out with hiring other than putting in a good word if I know someone personally. I hated interviewing applicants as much as sitting interviews myself.
There’s not a great way to find out in fifteen to thirty minutes if someone is competent or a good fit so we resort to asking these nonsense questions because it’s the done thing. It’s a bit like Agile. There’s no good reason to do it, so we just do it until someone comes up with something else to do.
I do pretty well at interviews these days if only because I’m relaxed (contracts are usually short-to-mid term so I have a very “it’s just one gig” attitude and tend to crack jokes and ask about the team rather than the project - I’m pretty sure I can do the work, I’m not sure we’ll get along 8 hours a day so that’s more important for me to find out).
To your question, if you’re thinking about throwing in the towel - I wouldn’t if it’s just because of a problem of interviewing.
You have experience and are therefore clearly capable - but life’s short and as you’ve said it’s not your passion you might be happier doing something else.
So the best answer I could give you brings you back to where you started I’m afraid:
Only if you want to.