Taking a step back, we rely on countless systems every day where
some reasonable proof-of-personhood is critical for the application, yet our countermeasures are best-effort in-house fraud prevention relying on security-by-obscurity. I suspect virtually all systems of today are ripe for exploitation in a rapidly evolving cat-and-mouse game. We’re in the early stages of a dirt cheap, sophisticated AI content generation revolution. Yet, our rhetoric suggests that these are minor, isolated issues with specific companies.
I’m wondering if it’s time to properly categorize these attacks as a separate and omnipresent danger to computer systems that process and distribute human input, and take it seriously. (Is there already a term for this? Sybil-attacks?) Are security researchers or academia largely overlooking an elephant in the room? Maybe I live under a rock, but I haven’t seen any interesting movement in analyzing, much less mitigating, these threats systematically.
Specifically, I fear the huge boost in human-imitation AI will shift the scales in favor of gray/black actors, due to reduced general trust-levels (as if they weren’t already low). If fake reviews cost $5 today, but $.01 tomorrow, there’s simply no way our dams will hold to that pressure. We really need some answers on how to navigate the upcoming sea of bullshit, or we’ll regress quickly into a world that’s disorienting at best, for good actors, and offers plenty exploitation by bad actors.