Bugs in our pockets: the risks of client-side scanning

"We didn't violate your privacy because we bought the information from a private company that violated your privacy. But it's ok, because you clicked a button, or signed a EULA."

That's the legal justification behind law enforcement fusion centers gathering, deanonymizing, and sharing data harvested by adtech firms, device manufacturers, and service providers.

It's a gotcha with the same intellectual weight as "I know you are but what am I?"

We need legislation with consequences lethal to companiesthat violate privacy.

Something like this - Every individual variable that a company wants to obtain from a person should be consensual with no option to select all, and the data and permission should be ephemeral. At any time a person should be able to inspect, delete, or allow continued possession of private data. They should be able to allow or deny sale or transfer of the data, and any recipient of the data must confirm permissions before taking receipt. Any algorithm or software or human analysis of data must be public and transparent. A record of any decision or business logic involving private data must be kept, and that record becomes private data, subject to the same constraints.

Stealing someone's identity should have a mandatory minimum of 2 years of community service, and total loss of opt in privileges for 5 years. No free web services or social media if you fuck around with someone else's privacy. Violations result in fines, paid to the victim and more community service.

A business caught abusing private data is subjected to a fine of 5% of company net worth per day. Half of the fine goes to the regulatory bureau, half to the victims.

Law enforcement must obtain warrants specific to known individuals - no geofencing or search term fishing expeditions. Digital data is subject to the same 4th amendment protections as physical papers and property.

Leaks would mean the end of an organization. If a company can't protect private data, then it can't participate in collecting it.

I'm sure there are flaws, but the gist of this seems a good starting outline. Anything less won't solve the problems and there should still be a mechanism for consensual participation in data markets. This would nuke credit bureaus, rein in isps and big tech abuses.

Does this count for something like AV, too? I grew up in a world where AV and anti malware only worked offline / client-side. What about spam filters and AV on mail servers? I often hear a commercial for Crowdstrike on Darknet Diaries podcast, which apparently is some kind of combination of a SIEM and ML (though that might be marketing). Would that be the panacea according to this paper? Or is this specific about a client-server model where the server hosts the data? Because the solution to that is rather simple 1) FOSS client 2) public-key cryptography where the private key does not enter the server. Sure, a cloud might make that illegal, but that's why the cloud is just someone else's server. Use your own instead.

While this is true in general, I think it's actually not true for Apple with respect to iPhotos (possibility of true end to end encryption), since they also make the photo processing software and the camera itself. The sensor data needs to be rendered to a file before it is even possible to encrypt it, so Apple could capture and scan that if they wanted to. You can't encrypt light waves before they hit the physical sensors.

Of course, the same is true of messaging. Apple owns the keyboard software and nothing stops them from putting a keylogger in to capture text before it ever hits the messaging app that encrypts it (and it pretty much needs to have one for predictive text to be possible).

They obviously can and arguably should choose to ignore the data streams before they hit the network clients and can be encrypted, but the capability will always be there.

I think part of the issue here is Apple owning all of the hardware, the OS, and the network client. People don't want to trust network clients, but you have no choice but to trust the OS and hardware vendors. If you don't trust them, your only option for guaranteed private communication is to not use computers. You either need to resort to the organized crime/terrorist model of using hand carry via couriers who credibly believe you'll kill them if they rat you out, or the military model of building your own communications devices.

The free lunch argument is not satisfactory because no matter the price of what you buy, the lunch is never paid for.

> I can see both sides

There is the side that says "we should know everything, because we own you" and the side that says "I prefer not to be owned". There is no good-faith way to see both sides of that, that is just avoiding a hard question because it is convenient.