I have a lot of thoughts about this particular topic, probably ideas that are unpopular and definitely go against the grain, at least in the way the Bay Area operates.
In my opinion, if you bring a candidate on-site for a 4+ hour interview and you ask them to write code, you've failed to properly vet the top of your recruitment funnel. As a software engineer, writing code is the BARE MINIMUM of your job. Imagine hiring a construction worker by merely asking them to hammer a nail for 4 hours.
You should have a thorough understanding, through various means such as phone screens, code samples, portfolio work, and simple tech challenge, to determine whether they can write code before you even bring them on-site. Once they're on-site, you should quickly verify that they can actually do what you previously learned that they can do. I'm talking 15-20 minutes worth of time. The remaining time should be 100% on the "soft skills" and other things mentioned, including but not limited to my big criteria: "Can you quickly get up to speed and learn the things you don't already know?" and "Do I want to sit next to you for the next 6 months of my work-life?"
If your on-site interview includes 45 minutes of coding (much less multiple sets of 45 minutes of coding!), you're doing it wrong. Very wrong.
Keep in mind you've asked your candidate to effectively take a day off work from their current job. They're calling in sick, taking a vacation day, or somehow lying to their boss and taking a risk of not being in their office when they're in yours for the interview. If they're smart, they've lined up multiple interviews with multiple companies, because putting all their eggs in your basket is just a terrible idea. Imagine having to take four days off to interview with four companies. And they probably aren't four days in a row, so you can't just pretend to have the bubonic plague or something.
You're asking them for a huge commitment of their time. And if you're willing to kick out a candidate because they didn't know how to invert a binary tree, then you should have found that out before they even walked in the door and saved everyone a whole bunch of time.
IMO, an interview should be a max of 4 hours. Any more than that and the candidate is simply repeating themselves over and over to different people, potentially solving the same tech problems (that should have been evaluated beforehand), and generally making it a huge waste of time.
My two cents. Happy to debate these ideas with anyone interested.