Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It's virtually impossible to read old iMessages and they take up tons of storage

keydiscussions.com

Post reply on HN
Print view View original

171–180 of 276 posts

Re: It's virtually impossible to read old iMessages and they take up tons of storage

#171

Post by prpl »

prpl wrote: post #160
judge2020 wrote: post #141

Earlier quoted context omitted.

Did your work turn into some potentially sharable shell scripts, perhaps? Would be a useful tool to avoid having to store messages in iCloud.

I have not used it for a very long time, so I can't comment to how well it works. That said, this is it: https://gist.github.com/brianv0/35f36a32366a2c34be8d (You need an unencrypted backup directory to run it over) Note: It looks like the flat directory structure of the backup is now 256 dirs prefixed by first two characters of file name. I'm not sure, but a small change to the name "sha_file" may be needed to just…

couple of extra hints:

Python 2 but can be ported forward without much effort. Manifest.mbdb is now Manifest.plist (there's a sqlite thing now too). timestamp sql might not work so great, would need to check query.

Re: It's virtually impossible to read old iMessages and they take up tons of storage

#172

Post by b3morales »

ascar wrote: post #162
CoolGuySteve wrote: post #157

Earlier quoted context omitted.

I've built an iMessage chat.db reader as well. To me the most astounding thing is that chat.db is unencrypted sqlite. Anyone who can see your home folder can read all your private chats or upload the chat.db to be read somewhere else. It's impressively insecure.

Reading this after GP called it reverse engineering makes me feel he should've use different terminology.

Why would a lack of encryption make it not reverse engineering? There's a database schema, it was engineered* and used to build a database. They went from the live data back to the schema.

*For some definitions of "engineered", apparently.

Re: It's virtually impossible to read old iMessages and they take up tons of storage

#173

Post by b3morales »

xoa wrote: post #117
KishanBagaria wrote: post #88

Earlier quoted context omitted.

Yes, with a catch: it requires an always on Mac (which can be in the cloud) or a jailbroken iOS device. We plan to make it work without the catch. I've worked with 10+ messaging platforms so far and Apple's protocol is the most obfuscated and complicated to reverse engineer. Apple has invested millions of dollars to make the iMessage protocol super hard to reverse engineer. It's how they sell tons of iPhones after al…

Looks like a super cool project. I'm always happy to see new client-side software for communication protocols written that aims to improve upon built-in! And iMessages in particular could certainly use it. With the demise of iTunes, it now feels like iMessages is probably one of the most crufty-but-heavily-utilized user facing pieces of software Apple puts out. Maybe they have some long term plans to refactor it ala…

> one of the most crufty-but-heavily-utilized user facing pieces of software Apple puts out. Maybe they have some long term plans to refactor it ala iTunes but yeesh.

The Messages app on Mac was rewritten for Big Sur (last year). It gained some features but also some bugs in the process.

Re: It's virtually impossible to read old iMessages and they take up tons of storage

#174

Post by shepherdjerred »

koolba wrote: post #120
chmaynard wrote: post #66

Earlier quoted context omitted.

That's an undocumented implementation detail. It might change without warning at any time. Are you seriously recommending this as a solution for users who want to be able to read past messages?

I’d imagine it’d be difficult for Apple to migrate to anything substantially different as it’d require rewriting a potentially multi GB file, the space for which many users would not have. It’s effectively a massive, distributed, database migration.

Database formats change all of the time. There's no technical challenge here that hasn't been solved before.

Re: It's virtually impossible to read old iMessages and they take up tons of storage

#175

Post by crossroadsguy »

If Apple is focusing on “services” now, as numerous tech/biz pundits have talking about for quite some time now, it doesn’t show in their tech and software offerings, except for the gloss in the presentations and the painfully obvious exaggerated wordings.

With every iOS and Mac OS release I’ve been hoping they’d fix iMessage, iCloud, App Store search, app discovery, extension ecosystem for browsers, or my iPhone taking those extra 7-8 phantom GBs (..iPhone Storage/Others) that everybody guesses could be some cache or logs or something magically else and speculates maybe one day it’ll go away but nobody knows for sure etc.

But no! It seems Apple has been saving these things so that after 10 years they get to say on stage “Now you can search apps like never before. We have named this patented revolutionary app discovery as iDiscover™. As simple as searching for apps!“

Or after another 5 years something like “Now you can read your iMessages in a more readable manner and it will truly sync now. Meet TrueSync™“ when their engineers will have a breakthrough after 17 years of backbreaking research.

I have always wondered how the engineers look in the mirror after hearing those pathetic stage ridiculousness.

Re: It's virtually impossible to read old iMessages and they take up tons of storage

#176

Post by germinalphrase »

4f77616973 wrote: post #109
spenvo wrote: post #28

Earlier quoted context omitted.

I actually did try emailing Tim Cook a few months ago, but I got bounce backs. I tried tim@apple.com and timcook@apple.com but the mailer daemons said the addresses weren't registered. It sounds like a crazy thing to do, but I remembered how Steve Jobs would reply to users' emails and read an article where Tim said he set aside time to read users' emails.[0] [0] - https://www.inc.com/business-insider/tim-cook-wakes-u…

Craig's email is [redacted]. The guy is very chill. It's better you email a software guy about software. Also, be sure to file a bug report / enhancement at https://www.apple.com/feedback/iphone.html

I’m somewhat surprised that acknowledging outside emails is a thing. I know unsolicited ideas/content/etc. is commonly rejected without review in other fields. Is this successful beyond identifying technical bugs?

Re: It's virtually impossible to read old iMessages and they take up tons of storage

#177

Post by jaredwatts »

sneak wrote: post #142

The complete iMessage history in your backups is also not e2e encrypted, so all of it is readable by Apple. They regularly turn this information over to US federal authorities without a warrant (so, no probable cause) too. Having such a huge trove of history of all of your communications saved that can be accessed by the police or a rogue Apple sysadmin at any time is something that I feel should even make the law-ab…

Messages are not included in backups if you use Messages in iCloud as the backup option. Messages in iCloud also offers end-to-end encryption.

Re: It's virtually impossible to read old iMessages and they take up tons of storage

#178

Post by forgotmypw17 »

I've had this sort of experience with many systems, which is why I decided to use .txt files as the base data store for my current project.

It is indexed in a relational database, and it is presented in a Web 1.1 frontend, but it all flows back through the text files, and they are always available for download, optionally in zip or tar.gz archives.

Re: It's virtually impossible to read old iMessages and they take up tons of storage

#179

Post by alexanderdmitri »

KishanBagaria wrote: post #65

Apple's chat.db has an esoteric schema owing to the fact they never designed it from the ground up and instead kept adding new columns and tables with each macOS release. This makes their queries super complicated with multiple joins. Once you have the schema figured out, it's dead easy to build a third-party client that works better than the official one. Even search works great with a simple LIKE query but Apple re…

iPhone is the only apple device I own but it crashes it can't go too far back into my messaging history for any one individual too far.

Usually once I scroll to ~5 months back with people I text regularly the iMessage app just straight crashes ... it's a little absurd. To top it off, every couple pages it gives a loading icon and introduces severe latency for whatever reason. There don't seem to be any apps in their App Store that let you search and lookup old messages either.

This is actively biting me right now. Are there any solutions for accessing the iMessage/chat.db database on my phone or via iCloud from a Linux system (Ubuntu in my case)?

Re: It's virtually impossible to read old iMessages and they take up tons of storage

#180

Post by shitRETARDSsay »

dave_aiello wrote: post #87
cryptoz wrote: post #5

Earlier quoted context omitted.

Related, I emailed Tim Cook about iPhone sensor future plans, and a high-level exec of like 25+ years wrote back to me within a week. Apple is remarkably good at getting back to cold emails to top execs.

I concur with that. I had a problem that appeared to be a corner case with Apple Pay. I wrote a paper letter to Tim Cook about it, and within a week got contacted by a staff person in his office. He worked exclusively with me to identify issues I had not considered, and followed up on all of the direct effects and side effects until they were resolved.

Seconded, I wrote about an issue I encountered on facetime and I was sent many iPhones to test how widespread the problem was.
Post reply on HN