Typically an interview question should be hard enough to test the limits of any candidate. The goal is to see how someone thinks and how honest they are when faced with a technical challenge, not what they were able to memorize.
With that in mind, take heart if the interviews have been sometimes off the mark. Most interviewers were just pulled from some normal day to ask some questions and often unconsciously see it as an opportunity to show off a bit. They have the keys and you need them.
That said, if you do not feel passionate, I would recommend playing around with some other areas like frontend or ML at home, on your own time, and see if any of them click.