This was an effective method to screen out applicants who didn't have the basic coding skills to align with their stated resume experience. And there were a decent number of these.
We did further development project exercises later in the process that took about 15 hours. We paid the candidates for this time, even those that didn't pass. It was also an effective screening tool.
All our exercises were very "real world." In the candidates own development environment and having been given instructions on how to prepare. They also have access to whatever Internet resources they want while doing the exercises. If they can't do the exercises, they can't do the job.
I know there are mixed opinions on this and I feel for candidates who have to invest a ton of time in exercises like this. But I can't imagine trying to hire without visibility into how they execute relatively basic software development tasks.
I think employers can and should structure the process so the time investment is minimal upfront and only increases as both parties have gotten to know each other and want to proceed.