Live data from Hacker News

Cleaning up my 200GB iCloud with some JavaScript

andykong.org

91–100 of 192 posts

Re: Cleaning up my 200GB iCloud with some JavaScript

#91

Photos.app doesn't show file size… so I thought, why not build a little Photos.app extension or a separate app that queries for large files? Turns out that the API doesn't expose "file size", at least I didn't find a straight-forward way. I think that all "photos" or "videos" are just a view of the underlying "photo or video object". If you crop a video, the full-size video will remain. Only if you export the video,…

[deleted]

Re: Cleaning up my 200GB iCloud with some JavaScript

#92

Photos.app doesn't show file size… so I thought, why not build a little Photos.app extension or a separate app that queries for large files? Turns out that the API doesn't expose "file size", at least I didn't find a straight-forward way. I think that all "photos" or "videos" are just a view of the underlying "photo or video object". If you crop a video, the full-size video will remain. Only if you export the video,…

this is the best plausible reason for why the file size differs. I've also noticed some unepxected media restoration (thought I cropped/edited a video, it is still there in full length and resolution).

This also helps explain why my iPhone storage always seems to be at its limits, despite my obsessive management.

Anyway, we all know foss unix ftw :P

Re: Cleaning up my 200GB iCloud with some JavaScript

#93
post #69

Earlier quoted context omitted.

>So as a consumer, do I just wave my hands and keep throwing more and more money at the problem, Yes. I can go and buy 1TB of Microsoft OneDrive or 2TB of Google Drive for less than a Franklin a year, and most people won't even need 1TB let alone 2TB. Both Microsoft and Google also offer 100GB plans for a Jackson a year, which is what I purchase myself. The average person can get by paying a Washington per month to A…

Now, consider a world where you are making $42,000 a year. That's 20 dollars an hour and a very common wage. How do they handle the same when there are so many competing Franklins for them?

bump this, and the implicit vendor lockin that this ideology creates

Re: Cleaning up my 200GB iCloud with some JavaScript

#94
post #9

I kind of hate opaque cloud solutions like this, because their storage usage is inscrutable and their interfaces are basically never optimized to help you clear things out. Whether that’s intentional and malicious or just not a priority, I don’t really care, I just want to be in control of the stuff I’m using. I have a very similar issue with iMessage in iCloud, and unlike photos, there’s no web interface. You have t…

The opaqueness is definitely a problem. My iMessage database has a corrupted spot somewhere in the message history; if I accidentally make the Messages app read that spot (via searching or scrolling back through old conversations) it'll silently start maxing out the CPU and burning energy for no reason. There is no error message, nothing obviously wrong-looking, and as far as I know everything works.

Hey, this sounds like an interesting problem. I am always looking for edge cases to test, if you have time would you mind checking if https://github.com/ReagentX/imessage-exporter works for you and if it crashes in that spot?

Re: Cleaning up my 200GB iCloud with some JavaScript

#95

The thing I detest the most is the weird tier that jumps from 200GB to 2TB. There’s no way to pay for incremental storage for 500GB or 1TB, which would be perfect for many families. In today’s age where storage is a commodity, this ought to be priced per GB used.

That way they can subsidize the free tier

Re: Cleaning up my 200GB iCloud with some JavaScript

#96

I kind of hate opaque cloud solutions like this, because their storage usage is inscrutable and their interfaces are basically never optimized to help you clear things out. Whether that’s intentional and malicious or just not a priority, I don’t really care, I just want to be in control of the stuff I’m using. I have a very similar issue with iMessage in iCloud, and unlike photos, there’s no web interface. You have t…

Not iCloud but Google makes it easy to drill down into how your storage quota is being used - https://one.google.com/storage

Re: Cleaning up my 200GB iCloud with some JavaScript

#97

I kind of hate opaque cloud solutions like this, because their storage usage is inscrutable and their interfaces are basically never optimized to help you clear things out. Whether that’s intentional and malicious or just not a priority, I don’t really care, I just want to be in control of the stuff I’m using. I have a very similar issue with iMessage in iCloud, and unlike photos, there’s no web interface. You have t…

The opaqueness is malicious. Apple offers a free iCloud account but no method of cleaning up space. Normal usage of a phone with all optional features disabled will eventually fill up the free tier space.

The user’s options are quit the ecosystem or pay up. Apple’s lock-in monopoly means they get to shove their customers into another monthly fee.

This is a classic example of the harm done to consumers by allowing anticompetitive practices. (The consumer has no choice but to pay)

When the US government gets off their butts and investigates Apple for antitrust, this will be one of the key findings. The fine should be some multiple of the amount of money they have illegally extracted from their customers.

Re: Cleaning up my 200GB iCloud with some JavaScript

#98
post #69
post #65

Earlier quoted context omitted.

Differential backups or any sort of versioning seemed like one of the most obvious culprits (that and or total redundant storage to preserve the file) but the issue with all of this is it’s entirely opaque. Ultimately you’re increasingly tethered to some service for your storage that you pay for periodically based on total storage yet you have little-to-no information how to best optimize that storage if you want to…

>So as a consumer, do I just wave my hands and keep throwing more and more money at the problem, Yes. I can go and buy 1TB of Microsoft OneDrive or 2TB of Google Drive for less than a Franklin a year, and most people won't even need 1TB let alone 2TB. Both Microsoft and Google also offer 100GB plans for a Jackson a year, which is what I purchase myself. The average person can get by paying a Washington per month to A…

As someone who has a Proxmox cluster at home (storage on RAIDZ, hot backups with PBS, cold backups on external HDDs) I literally recommend cloud storage for most people that ask me about backup solutions as it's simply not worth it for the average user. Those people already have all their data in the cloud anyways and share it on Facebook et al and they don't really care about the privacy side.

Remember it's not just buying the equipment, it's maintaining and understand it as well (e.g. I have to be familar with how ZFS works, how to restore a failed node, write some scripts, etc.). And with every backup solution you also need to be familar with the restoration process and test it occassionally to make sure it actually works as expected.

Re: Cleaning up my 200GB iCloud with some JavaScript

#99

I kind of hate opaque cloud solutions like this, because their storage usage is inscrutable and their interfaces are basically never optimized to help you clear things out. Whether that’s intentional and malicious or just not a priority, I don’t really care, I just want to be in control of the stuff I’m using. I have a very similar issue with iMessage in iCloud, and unlike photos, there’s no web interface. You have t…

[deleted]

Re: Cleaning up my 200GB iCloud with some JavaScript

#100
post #69
post #65

Earlier quoted context omitted.

Differential backups or any sort of versioning seemed like one of the most obvious culprits (that and or total redundant storage to preserve the file) but the issue with all of this is it’s entirely opaque. Ultimately you’re increasingly tethered to some service for your storage that you pay for periodically based on total storage yet you have little-to-no information how to best optimize that storage if you want to…

>So as a consumer, do I just wave my hands and keep throwing more and more money at the problem, Yes. I can go and buy 1TB of Microsoft OneDrive or 2TB of Google Drive for less than a Franklin a year, and most people won't even need 1TB let alone 2TB. Both Microsoft and Google also offer 100GB plans for a Jackson a year, which is what I purchase myself. The average person can get by paying a Washington per month to A…

"640KB is more than anyone will ever need"

It's a little absurd to think people don't need more than 2TB - especially on HN. Gamers will likely have 2TB in games alone, videographers often have many TBs of videos and photos from weddings and events in their life, many that care about health may have a few TB in genomic data mirrored on their computers to analyze, etc.

I would imagine it's hard to find people that wouldn't have TBs of data, if they were allowed to do so. The reason many people don't have TBs of data is they're limited by these exact companies you're claiming 'solve the problem' by offering limited storage.

It is notable however, that having better tools to organize, deduplicate, and compress data would be helpful to reduce some of the size of data that many people have. Over the years I've noticed my family will have multiple tar.gz archives, zip archives, etc, which (after extraction/unencryption) will share 20% files here, 10% files there, a 4kb jpg that's the same as a 100MB PNG here and there, etc. So yes, those 10TB archives may end up being 5TB if someone spent the time to really comb over, understand, make good decisions, and organize that data. But I have not yet seen anything that can scratch that surface yet, other than perhaps https://github.com/jjuliano/aifiles - but I won't use it until it's local only and has guarantees not to destroy data without explicit permission. An overlay filesystem that shows compression/deduplication with LLM capability like aifiles is probably the best option here.

However, I wouldn't imagine that most people's life data is less than 2TB even with all of this - it's mostly imposed as an artificial constraint by these companies.

Post reply on HN