It's scary, because CSAM (child sexual abuse material) is very, very broad, and quite vague in its definition¹. My three year old son stayed with my parents for a week, and he was being taught to swim in their swimming pool. Of course, I get sent pictures and clips of his progress by a proud grandma, and of course, him being a toddler, he felt swimming trunks to be completely optional (and of course, they are at his…
You're probably not the first person to ask the question of "but what about parents taking pictures of their naked kids in a pool", I'm sure they plan on having some way of dealing with it if anything to lighten their workload. That doesn't mean it will be perfect but no matter how jaded we all are, nobody wants to start falsely accusing innocent people.
The Dutch Toeslagenaffaire¹ was a massive failure of using discriminating algorithms to judge if tax payers were defrauding the state (most of those marked as fraudsters were innocent and many lost jobs, housing, and hundreds even had their children placed into foster care (!) due to the cascading effects caused by being on this list).
The British Post Office scandal² is another famous case that will be familiar to those from the UK.
1: https://news.ycombinator.com/item?id=31013994
2: https://en.wikipedia.org/wiki/British_Post_Office_scandal