It's such a weird trick that I love it!
Unlimited Google Drive storage by splitting binary files into base64
111–120 of 251 posts
Re: Unlimited Google Drive storage by splitting binary files into base64
#112I don’t know anyone remember but some years ago I remember seeing a file compressed from 1GB to 1mb. And I was amazed.
On the edonkey network, the file size would be reported raw but the clients could compress and transfer chunks to each other. Some guy had created an empty IL-2 sturmovik iso and seeded it. We lived at a government facility with ill-policed high speed (for the time) internet but even then I knew that I didn’t have a 400 Mbps connection. Maybe 2002/2003.
The whole thing only transferred a few kB. It looked like an entire disc though.
Re: Unlimited Google Drive storage by splitting binary files into base64
#113Earlier quoted context omitted.
I love that this exists
I want to watch some! Got any urls?
the readme has an example: https://www.youtube.com/watch?v=Fmm1AeYmbNU
Re: Unlimited Google Drive storage by splitting binary files into base64
#114I don’t know anyone remember but some years ago I remember seeing a file compressed from 1GB to 1mb. And I was amazed.
42.zip?
even more dangerous for bot
https://rehmann.co/blog/10-gb-27-kb-gzip-file-present-http-s...
Re: Unlimited Google Drive storage by splitting binary files into base64
#115Honestly this isn't ground breaking, we have been using BASE64 to convert binary to ASCII as a way of "sharing" files all the way to USENET days. While applications like these make it easy for the masses to participate in the idea, they don't bring anything new to the table.
That all said, this is really cool from a design perspective and I poured over the code learned a lot.
Re: Unlimited Google Drive storage by splitting binary files into base64
#116ELI5 please?
The script split file into small base64 chunks that are stored into "documents" (mime type: application/vnd.google-apps.document ) that apparently don't count against google drive quota.
Re: Unlimited Google Drive storage by splitting binary files into base64
#117Earlier quoted context omitted.
I have been doing it for a long time, the hardest for me is all the registered users I have around the web linked to the email. After a few years of changing each one that mattered I finally get close to zero mail on gmail. Search I moved to ddg, that was the easy one. Android can work fine with just f-droid since I noticed I rarely even use the store any more and I need just a few essential apps. For storage, I tend…
I used shoebox as a great alternative to Google photos. The problem is they just shut down.
As I wrote that comment I went on another small search as I do every so often and I found Canon Irista and I have to say, I am impressed. The sync seems to work fine, it's pretty fast and the UI both of the website and the app is pretty solid. I suggest giving it a try if you are on the lookout for a new photo hosting service.
Re: Unlimited Google Drive storage by splitting binary files into base64
#118I wonder if this could be used to create a P2P network like bit torrent except trackers point to blocks at google doc urls instead of peers/seeds
I discovered that a lot of pirate stream sites are already doing something similar (but not exact) to this.
They store fragments of movies (rather than the full videos) in Google Drive files and then combine them together during playback. Each fragment could then be copied and mirrored across different accounts, so if any are taken down they can just switch to another copy. Pretty clever (albeit abusive) solution for free bandwidth.
Re: Unlimited Google Drive storage by splitting binary files into base64
#119Earlier quoted context omitted.
There's a good chance they won't. For privacy reasons, engineers can't just start peering at your files. They'd have to write a base64 detectors and automate the detection and banning of the accounts without the engineer ever seeing your files. Any bugs in that code, and they'll ban innocent people.
Wouldn't it be simpler to just set a generous limit for the number of Google Docs files? Say, 15 thousand?
I bet implementing such a limit would be 3 or more months of engineering effort.
Think about the difficulties. It has to take into account shared directories. It has to know about systems which auto-create documents (like results sheets for Google forms). It has to work with gsuite sysadmins who need to take ownership of files from deleted accounts. The UI to show when you have hit the limit has to be designed. And the support team has to be trained on how to resolve that error. And you're going to have to get that error message translated into 30 languages. Users already over the limit are going to be unhappy - are you going to write extra code to give them a grace period? How will you notify them of that? Will you have a whitelist of users allowed to go over the limit? How will you keep the whitelist updated and deployed to the servers? Who will have access to add/remove people from the whitelist?
The actual system itself has race conditions:. What if that 15000th file was simultaneously created in the USA and Europe? There is no way to prevent that without a cross-ocean latency penalty. Do you want to pay that penalty for every document creation? How do you deal with a net-split where cross ocean network traffic is delayed?
Finally, how will you monitor it? Will you have logging and alerting for users hitting the limit? Will there be an emergency override for engineers to remove the limit if necessary?
At big-web-service scale, simple engineering problems become complex problems fast...
Re: Unlimited Google Drive storage by splitting binary files into base64
#120So, are the +- 700 kB files too small to register as taking up any space?