Show HN: Apple Notes Liberator – Extract Notes.app Data and Save It as JSON

I am excited to see more solutions that work with Apple Notes database directly. This is my go-to application to capture unstructured memos while researching various topics. Last year I got carried away and built a simple site generator for Notes [0]. As a fun exercise I implemented ORM models to query Sqlite database in Ruby:

  User::Account.server(database: "tmp/test.sqlite").first.
    folders_dataset.offset(1).first
    notes.
    map(&:attachments).flatten.
    find { |el| el.is_a?(User::ObjectTypes::EmbeddedPublicJpeg) &&
      el.height > 700 &&
      el.summary[/animal/i] }.
    media_filepath
 => "Accounts//Media/F669B83A-0C47-408E-9645-015737F3B11F/Pasted Graphic.jpg"

Apple Notes' database structure is a fascinating product story. Notes started as a simple note-taking application that stored its content as raw HTML in a few simple tables. However, as the product grew and needed more features like collaboration, they had to make changes and add new columns and tables. They also implemented two flavors of CRDT-like algorithms that evolved over time: one for plaintext chunks and the other one for embedded objects like tables. On top of that, every change that Apple makes must be backward compatible to support legacy devices and systems; but backward-compatibility was not always possible and users had to go through “Migrate your Notes” steps, but it is again a product story.

I wish that one day, someone from the development team would write an article about it.

[0] https://devlog.notespub.com/2022/08/site-generator-for-apple...

At https://support.apple.com/en-us/HT204055, Apple provides instructions for a bunch of stuff you might want to export:

  • iCloud Drive files
  • Photos and videos
  • Mail
  • Contacts
  • Calendar
  • Reminders
  • Safari
  • Notes
  • Voice Memos

And/or you can use https://privacy.apple.com/, which is basically Apple's version of Google Takeout.

I wrote a similar tool in Python a few weeks ago, which uses AppleScript to liberate the data and saves it to a SQLite database: https://datasette.io/tools/apple-notes-to-sqlite

I didn't actually know AppleScript before writing this tool... but it turned out ChatGPT knew it well enough to unblock me and provide me with exactly the code I needed to build the rest of the project! https://til.simonwillison.net/gpt3/chatgpt-applescript

Apple Notes Liberator creates a copy of the SQLite database and then runs queries against that directly to extract the data.

I chose not to do that (despite being all-in on SQLite for everything) because I worry about future changes to the software baking my script - I figured the AppleScript interface was less likely to have breaking changes in the future.

This is great! How do you account for updates to a note?

I’ve been building an archiver tool as well to convert notes into markdown and post to git using Apple Shortcuts.

The interop between almost all iOS data formats (notes, safari, calendar, etc.) the content graph[0] provides is really impressive.

[0]: https://support.apple.com/en-me/guide/shortcuts/apd4618db957...

I loved your post about the pelican! https://simonwillison.net/2020/May/21/dogsheep-photos/