Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unlimited Google Drive storage by splitting binary files into base64

github.com

Post reply on HN
Print view View original

231–240 of 251 posts

Re: Unlimited Google Drive storage by splitting binary files into base64

#231

Post by manigandham »

londons_explore wrote: post #68
sircastor wrote: post #46

Earlier quoted context omitted.

I think that depends on their tools and how they evaluate data usage. If the reporting states that the accounts are using very little storage because it's using the same measuring stick that the client does them it's invisible. The question comes up during an audit of the system when the disk usage doesn't match the report. Then again, if this is used by few people it may just look like a margin of error.

It'll more be that the Google docs "live editing" backends are expensive to use disk and memory wise. They store complete version history with each keystroke of a document. There's a good chance a megabyte of "document" costs Google a gigabyte of internal storage...

They don't store a complete version history. It just uses checkpoints in their timeline of real-time edits and computes the differences when you need them. Those deltas can also be compressed.

Re: Unlimited Google Drive storage by splitting binary files into base64

#233

Post by Someone »

jerf wrote: post #210
imtringued wrote: post #204

Earlier quoted context omitted.

They can just throttle access to Google documents to something like 4 GB per hour and then block obvious abuses. If people start encoding bits as English sentences they are reducing the amount of useful data they can download within an hour which is exactly what you want.

"They can just throttle access to Google documents to something like 4 GB per hour" No, that's not likely to work. I'm sure there's far more legitimate users using 4GB of documents per hour than abusers right now. You have to remember things like bulk downloading, bulk scanning, bulk backing-up, shared automated accounts doing all sorts of legit things, etc. are all legitimate use cases. You can't just throw out all…

I would guess those enterprise users pay for Google docs, and could be exempted from throttling on that basis.

If they don’t, Google wouldn’t lose much by throttling them, would they?

Re: Unlimited Google Drive storage by splitting binary files into base64

#234

Post by 300bps »

xamuel wrote: post #183
romanhn wrote: post #63

Earlier quoted context omitted.

That reminds me of the first time I accessed World Wide Web. Back in '96 I was browsing a computer magazine and happened upon a listing of useful mailing lists, one of which returned the contents of web pages for a requested HTTP address. Same magazine had an install CD for the free Juno email service. Being a teenager, the first web page I ever requested was www.doom.com, which returned a gibberish of text to Juno's…

It's really sad thinking how kids these days totally miss the wonder of the early internet. In my case, it was at the public library. The lone internet computer was constantly booked. But by watching over a library clerk's shoulder, I was able to see the password needed to unlock the text-based library catalog terminals (which terminals were plentiful and always available). (My parents worked at the library, or else…

It's really sad thinking how kids these days totally miss the wonder of the early internet.

And before that, the wonder of bulletin board systems. I got my first modem in 1985!

Re: Unlimited Google Drive storage by splitting binary files into base64

#235

Post by acuozzo »

baroffoos wrote: post #12

Has anyone actually tried storing a large amount of data like this? I feel like creating a new google account and using it as a backup for a 300gb folder I have.

Yes. It's called: Post to alt.binaries.* on Usenet.

It's effectively the same thing under the hood. Binaries are split and converted to text using yEnc (or base64, et al.) and uploaded as "articles". An XML file containing all of the message-IDs (an "NZB") is uploaded as well so that the file can be found, downloaded, and reassembled in the right order.

This form of binary distribution has been around since the '80s if you change some of the technical details; e.g. using UUencode rather than yEnc.

Spend $5 for a 3-day unlimited Usenet account with e.g. UsenetServer.com and upload it.

If you want it to stay up, then make another account in 3925 days (the retention period), download it, and then reupload it for another 10+ years of storage.

Re: Unlimited Google Drive storage by splitting binary files into base64

#236

Post by binwiederhier »

collinmanderson wrote: post #220
binwiederhier wrote: post #19

In the same spirit, I made a few "just for fun" plugins for my (now abandoned) encrypted-arbitrary-storage Dropbox-like application Syncany: The Flickr plugin [1] stores data (deduped and encrypted before upload) as PNG images. This was great because Flickr gave you 1 TB of free image storage. This was actually super cool, because the overhead was really small. No base64. The SMTP/POP plugin [2] was even nastier. It…

I have a feeling PNG might work on Google Photos too, but I haven't tried it.

I can't remember if I tried, but it's important that you get the exact data back that you put in, which is why JPEG obviously won't work.

BMP is the easiest to encode/decode because it's literally a bitmap of RGB, no fancy compression and such, which, if you're storing arbitrary data is obviously not necessary.

PNG was trickier, because of its "chunks" and generally more structure. And compression.

Re: Unlimited Google Drive storage by splitting binary files into base64

#237

Post by binwiederhier »

collinmanderson wrote: post #219
lofties wrote: post #36

Very cool! About a year ago I had a similar idea, but to store arbitrary data in PNG chunks[1] and upload them to "unlimited" image hosts like IMGUR and Reddit. [1] http://blog.brian.jp/python/png/2016/07/07/file-fun-with-pyh...

I have a feeling PNG might work on Google Photos too, but I haven't tried it.

If you wanna give it a shot, try the code I linked here: https://news.ycombinator.com/item?id=19916126

Although if Picasa (predecessor to Google Photos) worked with BMP, it may be better to do that because it's much easier and more space efficient to encode arbitrary data in than PNG.

Re: Unlimited Google Drive storage by splitting binary files into base64

#238

Post by guitarbill »

LostJourneyman wrote: post #187
vageli wrote: post #164

Earlier quoted context omitted.

> [0] I use fastmail + custom domain, which works great, but you have to guard the domain very closely. What do you mean by guarding the domain? To prevent large volumes of spam?

I think OP means that you have to make sure you don't forget to/neglect to renew it and make sure you don't accidentally lose the domain for any reason.

spot on, basically you now have to worry about the domain being lost or hijacked also. for me, the flexibility to change email providers behind a domain is worth it though

Re: Unlimited Google Drive storage by splitting binary files into base64

#239

Post by warp_factor »

iKevinShah wrote: post #75
warp_factor wrote: post #73

Earlier quoted context omitted.

The fact that this is even a remote possibility should worry everyone of the ugly monopoly that Google became. I found myself in a similar situation a couple months ago. An android App falsely charged me on the Play store. After trying to contact Google for multiple weeks I gave up and disputed the charge on my credit card. This resulted in Google coming after me for 8.99$ and threatening me to close all my Google ac…

> I decided to move everything OUT of Google Have you been successful in it? Any guidelines / tips? How hard is it?

Late reply but here is what I did:

- Bought a new domain name and moved my mails in fastmail. I have been super happy with it so far.

- My Gmail address is now only for spam or very low importance emails.

- All my Pixel pictures are still uploaded to Google Photos, but I backup everything once a month or so.

- I don't use Google Drive for anything anymore. I have an Evernote account and a Dropbox account.

- Completely switchecd to DDG and Firefox.

- I'm still using my Pixel2 as of now but my next upgrade will be an IPhone, or a rooted Google-Independent Android phone.

Re: Unlimited Google Drive storage by splitting binary files into base64

#240

Post by warp_factor »

rahimnathwani wrote: post #181
warp_factor wrote: post #73

Earlier quoted context omitted.

The fact that this is even a remote possibility should worry everyone of the ugly monopoly that Google became. I found myself in a similar situation a couple months ago. An android App falsely charged me on the Play store. After trying to contact Google for multiple weeks I gave up and disputed the charge on my credit card. This resulted in Google coming after me for 8.99$ and threatening me to close all my Google ac…

"An android App falsely charged me on the Play store." I'm curious to know how this happened. Would you mind sharing more info? As I understand it, the only way for an app to 'charge you on the play store' is to: 1) Be a paid app (in which case you pay before the app starts installing), or 2) via in-app purchases, which are handled by the app initiating the IAP, and then Play services taking over to ask for confirmat…

Sure, I might have hit a corner case but I made an in-app purchase for a one year subscription for a service.

After using the app for a couple of days and restarting the phone, the app seemed to hit a bug and behave like if I didn't buy the subscription, prompting me to buy another subscription which I did thinking that this would unblock the backend and somehow merge with the fact that I already had a subscription.

Unfortunately, Google Play charged me again for a subscription I already had. Both the app creator AND Google Play were difficult to join. The App creators never replied to any of my emails. Google Play got an automated support website that decided that "I was not eligible for a refund" and there was nothing I could do about it. It also seems to be impossible to contact a real human being to explain the situation.

Post reply on HN