Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

rareese.com

Post reply on HN
Print view View original

251–260 of 716 posts

Re: Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

#251

Post by ethin »

This "let's not back up .git folders" thing bit me too. I had reinstalled windows and thought "Eh, no big deal, I'll just restore my source code directory from Backblaze". But, of course, I'm that kind of SWE who tends to accumulate very large numbers of git repositories over time (think hundreds at least), some big, some small. Some are personal projects. Some are forks of others. But either way, I had no idea that Backblaze had decided, without my consent, to not back up .git directories. So, of course, imagine how shocked and dismayed I was when I discovered that I had a bunch of git repositories which had the files at the time they were backed up, but absolutely no actual git repo data, so I couldn't sync them. At all. After that, I permanently abandoned Backblaze and have migrated to IDrive E2 with Duplicati as the backup agent. Duplicati, at least, keeps everything except that which I tell it not to, and doesn't make arbitrary decisions on my behalf.

Edit: spelling errors and cleanup

Re: Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

#252

Post by Chaosvex »

Aerroon wrote: post #248
michaelbuckbee wrote: post #116

Earlier quoted context omitted.

Is there an example of a consumer facing SaaS that's been able to handle the "unlimited" in a way you'd consider positive?

Google and Youtube, especially Youtube.

YouTube is constantly reencoding videos to save space at the expense of older content looking like mud, so arguably even they're having their struggles.

Re: Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

#253

Post by tedivm »

Aerroon wrote: post #248
michaelbuckbee wrote: post #116

Earlier quoted context omitted.

Is there an example of a consumer facing SaaS that's been able to handle the "unlimited" in a way you'd consider positive?

Google and Youtube, especially Youtube.

Google does not have unlimited. I had to pay to increase my storage.

Re: Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

#254

Post by bayindirh »

Chaosvex wrote: post #250
bayindirh wrote: post #196

Earlier quoted context omitted.

> Arguably they should even when not in that mode, but it'll churn files repeatedly as you stream files in and out of local storage with the cloud provider. When you have a couple terabytes of data in that drive, is it acceptable to cycle all that data and use all that bandwidth and wear down your SSD at the same time? Also, high number of small files is a problem for these services. I have a large font collection in…

Then do it in memory, assuming those services allow you to read the files like that. It sounds like they do based on your other comments.

The problem is, downloading files and disk management is not in your control, that part is managed by the cloud client (dropbox, google drive, et. al) transparently. The application accessing the file is just waiting akin to waiting for a disk spin up.

The filesystem is a black box for these software since they don't know where a file resides. If you want control, you need to talk with every party, incl. the cloud provider, a-la rclone style.

Re: Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

#255

Post by ethin »

Dylan16807 wrote: post #199
bayindirh wrote: post #195

Earlier quoted context omitted.

> Unless it does something very weird it won't trigger all those files to download at the same time. That shouldn't be a worry. The moment you call read() (or fopen() or your favorite function), the download will be triggered. It's a hook sitting between you and the file. You can't ignore it. The only way to bypass it is to remount it over rclone or something and use "ls" and "lsd" functions to query filenames. Other…

Why would it use either of those on all the files at once? It should only be opening enough files to fill the upload buffer.

I think you might be confusing Backblaze reading files and how Dropbox/OneDrive/Nextcloud/etc. work. NC doesn't enable this by default (I don't think), but Windows calls it virtual file support. There is no avoiding filling the upload buffer, because Backblaze has zero control over how Dropbox downloads files. When Backblaze requests that a file be opened and read, Windows will ask Dropbox or whatever to open the file for it, and to read it. How that is done is up to whatever handles the virtual files. To Backblaze, your Dropbox folder is a normal directory with all that that entails, so Backblaze thinks that it can just zip through the directory and it'll read data from disk, even though that isn't really what's happening. I had to exclude my Nextcloud directory from my Duplicati backups for precisely this reason -- my Nextcloud is hosted on my server, and Duplicati was sending it so many requests it would cause my server to start sending back error 500s.

And no, my server isn't behind cloudflare, primarily because I don't have $200 to throw at them to allow me to proxy arbitrary TCP/UDP ports through their network, and I don't know how to tell CF "Hey, only proxy this traffick but let me handle everything else" (assuming that's even possible given that the usual flow is to put your entire domain behind them).

Re: Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

#256

Post by xmcp123 »

Hamuko wrote: post #115
XCSme wrote: post #63

Initially I thought this was about their B2 file versions/backups, where they keep older versions of your files.

B2 is not a backup service. It’s an object storage service.

Weird, because in the Reddit thread linked above they call themselves a backup service.

Re: Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

#257

Post by balderdash »

not helpful for non-mac users, but i really like the way arq separates the backup utility from the backup location. I feel like the the reason backblaze did this was to save money on "unlimited" storage and the associated complexity of cloud storage locations.

Re: Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

#259

Post by Dylan16807 »

jonhohle wrote: post #247
Dylan16807 wrote: post #233

Earlier quoted context omitted.

4 writes out of what, 3000? For something you'll need to do once or twice ever? It's fine. You might not even eat your whole Drive Write Per Day quota for the upload duration, let alone the entire month. > the technical limits of the technology that I had at home (PON for the time being) Isn't that usually symmetrical? Is yours not?

How do you know how often those files need to be backed up without reading them? Timestamps and sizes are not reliable, only content hashes. How do you get a content hash? You read the file.

Backblaze already trusts the modification date.

Re: Backblaze has stopped backing up OneDrive and Dropbox folders and maybe others

#260

Post by philjohn »

For those looking for something at a decent price for up to 5TB, take a look at JottaCloud, which is supported by rclone, and then you can layer restic on top for a complete backup solution.

JottaCloud is "unlimited" for $11.99 a month (your upload speed is throttled after 5TB).

I've been using them for a few years for backing up important files from my NAS (timemachine backups, Immich library, digitised VHS's, Proxmox Backup Server backups) and am sitting at about 3.5TB.

Post reply on HN