I built a system just like this at another company whose products or services you likely use often or everyday.
This pessimistic view assumes the worst about people like me who build these kinds of systems, as if we’re evil or corrupt or somehow doing this to take something from you.
In reality, data is stored in disparate systems, under the custodianship of different organizations. Once you can find everything and account for it, you need to query every single system - many systems which aren’t built for this kind of “on demand” workload. Then you need to parse the data, turn it into some kind of useful values, especially if the internal representation contains flags, enums, or other magic or pseudo values that wouldn’t be meaningful to anyone but the logic or programmer who wrote it. Systems go down. Things break. Pipelines get clogged. It’s one thing to build a god system that can decrypt, read, and perform etl on every application, table, db, or whatever storage used anywhere in your entire company. It’s exponentially harder to solve this problem when it’s all legacy integrations with shit that’s duct taped together and will easily tip over.
Now you have to do this at scale - except these systems have millions of lines of code and can’t just be rewritten into a solution that can handle hundreds or thousands or even tens of thousands of queries per second… not without a Herculean effort not even accounting for all the tribal knowledge that’s been lost on how the system is expected to work.
If 30 days is too long for you, essentially you’re wanting these companies to spend potentially hundreds of millions of dollars to rearchitect a significant chunk of their systems that were built prior to all these privacy laws coming online.
Honestly, the legal landscape changes often. Some of the law is open to interpretation. My own experience working in this are require working closely with a team of lawyers. Honestly, even the Staff Engineers in my larger org getting paid $700k a year would have preferred any other project but this.