So you believe that writers are not making the necessary effort and write using LLMs, so you then use an automatic LLM to filter it as a reader(because you don't care as a reader and don't want to make the effort manually). So this way there will be a lot of false positives like with school assignments. I think a better solution would be to have a network of people that you trust manually read and label texts instead…
Personally I think a web-of-trust Keybase-style solution could work... But buy-in is difficult... you'd need some sort of "seed" strategy to make the app useful alack of 100 (a huge number) trusted friends.