One other major downside of end-to-end encryption: everything has to be done client-side, and the server becomes very close to a dumb pipe and block storage. (The only parts the server can do anything with are those that aren’t encrypted, so the explicit goal of E2EE is to reduce the server to a dumb pipe and block storage.) Got ten gigabytes of stuff you want to search through? Without E2EE, the server can implement…
1. The idea that it’s impractical to download the data. We are talking about searching text notes. 10gb? That’s nearly 7 million pages of notes. Doesn’t seem a reasonable figure. Even hundreds of megs of text would require a lot of time to acquire and even that would rarely need to be downloaded in one go (new device).
2. The idea that tech that tends to live on the server is somehow magical. Search would take “a few seconds” on a “slow” phone processor you say. Phone professors are incredibly fast these days, but in any case, since when is text search slow? I feel like many developers treat text search as a scary magical black box but it’s rather straightforward. Read about inverted indexes and consider that the servers of the mid 90s were serving at least tens of thousands of text searchers apiece with processors probably slower than what’s in your phone. There are libraries you can use locally to make text search pretty trivial. Apple supports text search (spotlight) across the data and apps on your phone. It’s not rocket science.
3. The idea that anything other than conceptually perfect encryption is useless. “first-party end-to-end encryption is snake oil” - assuming you mean crypto where the user does not handle the keys directly and a first party causes them to be generated, first party e2ee describes some of the biggest privacy wins of the last 15 years - iMessage, FaceTime, Signal, WhatsApp.