Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cleaning up my 200GB iCloud with some JavaScript

andykong.org

Post reply on HN
Print view View original

181–190 of 192 posts

Re: Cleaning up my 200GB iCloud with some JavaScript

#181

Post by s3p »

InsomniacL wrote: post #25

I'm pretty sure that iCloud stores the full version and downloads a lower quality optimised version to your phone. If you open iPhone settings and browse to: > Apple ID > iCloud > Photos There is an option to 'Optimise iPhone Storage' which is enabled by default. This states: > If your phone is low on space, full resolution photos and videos are automatically replaced with smaller device sized versions. Full-Resoluti…

This isn't related.

This deals with the storage on the phone, which you can find in Settings > Manage storage. It has nothing to do with the storage in the cloud.

Re: Cleaning up my 200GB iCloud with some JavaScript

#182

Post by ezfe »

nytesky wrote: post #132
ezfe wrote: post #67

Earlier quoted context omitted.

iCloud Photos has deduplication

Yeah I ran it and shrunk my library by half. It was kinda scary since I had 100000 photos, I couldn’t really inspect the process

It literally shows each photo set it thinks are duplicates, you can literally manually verify

Re: Cleaning up my 200GB iCloud with some JavaScript

#183

Post by nytesky »

ezfe wrote: post #182
nytesky wrote: post #132

Earlier quoted context omitted.

Yeah I ran it and shrunk my library by half. It was kinda scary since I had 100000 photos, I couldn’t really inspect the process

It literally shows each photo set it thinks are duplicates, you can literally manually verify

I didn’t have time to manually verify 10s of 1000s of duplicates. I spot checked and it looked fine..

Re: Cleaning up my 200GB iCloud with some JavaScript

#184

Post by Groxx »

Groxx wrote: post #142
asdaq1312512 wrote: post #53

Photos.app doesn't show file size… so I thought, why not build a little Photos.app extension or a separate app that queries for large files? Turns out that the API doesn't expose "file size", at least I didn't find a straight-forward way. I think that all "photos" or "videos" are just a view of the underlying "photo or video object". If you crop a video, the full-size video will remain. Only if you export the video,…

Since you're willing to write since code apparently: have you checked the internal databases? They're just sqlite, moderately understandable at a glance last time I looked.

>write since code

"since" -> "some". ugh. I had a bad swiping day apparently.

Re: Cleaning up my 200GB iCloud with some JavaScript

#185

Post by theshrike79 »

mysteria wrote: post #98
Dalewyn wrote: post #69

Earlier quoted context omitted.

>So as a consumer, do I just wave my hands and keep throwing more and more money at the problem, Yes. I can go and buy 1TB of Microsoft OneDrive or 2TB of Google Drive for less than a Franklin a year, and most people won't even need 1TB let alone 2TB. Both Microsoft and Google also offer 100GB plans for a Jackson a year, which is what I purchase myself. The average person can get by paying a Washington per month to A…

As someone who has a Proxmox cluster at home (storage on RAIDZ, hot backups with PBS, cold backups on external HDDs) I literally recommend cloud storage for most people that ask me about backup solutions as it's simply not worth it for the average user. Those people already have all their data in the cloud anyways and share it on Facebook et al and they don't really care about the privacy side. Remember it's not just…

I've just upgraded from a Synology 4 disk setup to a custom-built Unraid system.

Wouldn't recommend it to Joe Normal. I've spent a good 5-6 days just rebuilding parity while switching disks. During which the server makes a godawful noise and the performance is degraded because of constant disk load.

It's great to have 40TB+ of storage right next to me though, the performance is great and I can run every self-hostable service imaginable with it.

But I still subscribe to Apple One for the whole family, it just works for every device (3-4 phones, 3 tablets, 2 laptops). I do have some backups running from those to the local system, but mostly it's for quick restores.

The only thing I'm adamant about is to keep your generated content for yourself. Don't trust FB, Instagram, Youtube or whatever to be the only storage of anything you create. Keep the master copy where you control it, publish it to other services.

Re: Cleaning up my 200GB iCloud with some JavaScript

#186

Post by InsomniacL »

wideroots wrote: post #174
InsomniacL wrote: post #25

I'm pretty sure that iCloud stores the full version and downloads a lower quality optimised version to your phone. If you open iPhone settings and browse to: > Apple ID > iCloud > Photos There is an option to 'Optimise iPhone Storage' which is enabled by default. This states: > If your phone is low on space, full resolution photos and videos are automatically replaced with smaller device sized versions. Full-Resoluti…

This doesn't explain the case when the author uploaded and downloaded a new file, and there was no big diff in file size. The file size diff only seems to be happening to the author's older video files.

It does explain it and likely adds more weight to it too.

Encoding additional videos takes a lot of processing power. When you upload a video to youtube, the higher-quality versions aren't available for sometime. There is likely a queuing system, maybe even a minimum weight time before a video is processed.

Re: Cleaning up my 200GB iCloud with some JavaScript

#187

Post by vel0city »

CitrusFruits wrote: post #165
vel0city wrote: post #102

Earlier quoted context omitted.

> Gamers will likely have 2TB in games alone If I were to install all the games on my steam account it would be many hundreds of terabytes of total storage. In the end I have about a terabyte of games on my computer. And of that 0 bytes are in my cloud storage. I've been an amateur photographer for over 15 years. I tend to curate the photos I keep, largely because I don't need 20+ pictures of the same scene. Its more…

You could argue all your games are in the cloud since of the hundreds of terabytes of games you have, you're only keeping 1TB of them on the computer.

Its even more than that; easily half to three-quarters of my gaming these days is on cloud gaming so I'm not even installing any content.

But in the end, that's not the same as my cloud storage where I'm being metered by my bytes.

Re: Cleaning up my 200GB iCloud with some JavaScript

#188

Post by rhettbull »

asdaq1312512 wrote: post #53

Photos.app doesn't show file size… so I thought, why not build a little Photos.app extension or a separate app that queries for large files? Turns out that the API doesn't expose "file size", at least I didn't find a straight-forward way. I think that all "photos" or "videos" are just a view of the underlying "photo or video object". If you crop a video, the full-size video will remain. Only if you export the video,…

https://github.com/RhetTbull/osxphotos will do this:

osxphotos query --min-size 100MB --add-to-album "Big Files"

Finds all photos, videos > 100MB and adds them to the album "Big files"

`osxphotos query --help` for more info or `osxphotos docs` to open docs in the browser.

Disclaimer: I'm the author.

Re: Cleaning up my 200GB iCloud with some JavaScript

#189

Post by rhettbull »

LVB wrote: post #170
asdaq1312512 wrote: post #155

Earlier quoted context omitted.

So it appears you can conveniently query the DB for filename (+directory), UUID, and (original) file size. But I can't find a good way of opening a Photo by filename or UUID. After all, the file might not be present locally, so opening it should go through the Photos.app. But once you call AppleScript to open the file, you might as well use the AppleScript to comb through the database like the script I linked to earl…

Check out this which I believe has that feature: https://github.com/RhetTbull/osxphotos

Yes, `osxphotos show UUID` will open the photo with UUID in the Photos app. (I'm the author of osxphotos)

Re: Cleaning up my 200GB iCloud with some JavaScript

#190

Post by rhettbull »

cianmm wrote: post #4

Interesting article. I have my own problem, that caused me to have to upgrade my iCloud plan, and thus may not be a high priority fix for Apple. If you shoot RAW+JPEG (not a super-rare thing to do, for photo enthusiasts) then Apple Photos links the two images. Which is useful, rather than having a bunch of kinda-duplicates littering your library you can easily toggle between RAW and JPEG. But this combining, along wi…

> Any method that I've found to clean them up (exporting the originals, deleting them from the library, and then re-importing the JPEGs only seems easiest) will lose all of the years of metadata that I've built up in the library.

The open source tool osxphotos (https://github.com/RhetTbull/osxphotos) can help with this. You can export the JPEG images while preserving metadata using the thrid-party exiftool utility:

`osxphotos export /path/to/export --has-raw --skip-raw --exiftool`

This exports all images that have a raw pair but skips the raw component then uses exiftool (https://exiftool.org/) to write the metadata (keywords, etc.) to the exported JPEG files. You can then re-import these into photos either by dragging them or by running `osxphotos import /path/to/export/*`

Both the export and import commands have many other options for controlling export directory, etc. `osxphotos help export` or `osxphotos docs` to open docs in browser. (Disclaimer: I'm the author of osxphotos)

Post reply on HN