Really neat, are you running the cloth simulation on the CPU or GPU? How many elements are in the simulated cloth? Good luck on future AR projects!
The limiting factor is actually not the sim, but the initial load time for the procedural geometry. I might try adding some subdivisions though..