Is there a cheap s3 compatible service that is less reliable? I dont want to pay for redudancy Eg its my backups I can handle a 3% chance that my data is lost as long as I find out about it.
Backblaze B2 Cloud Storage Now Has S3 Compatible APIs
71–80 of 293 posts
Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs
#72Earlier quoted context omitted.
For backups, mapping one file to one object rarely works well. Tools that use this strategy come with a long list of scenarios requiring expensive operations. For instance renaming a directory or changing a few bytes in a large file. On the other hand tools that don't view the object storage as a file system have way less gotchas. In my experience B2 + restic works really well.
That's a fair point but this is for my family photos and videos; in the event I die I'd rather the S3 bucket I hand over to my wife/kids in my last wishes look like a real filesystem rather than a bazillion blobs that require a special tool with a programmer's expertise to reassemble
May I ask which solution you ended up going for in this situation? (I'm trying to solve for myself too.)
Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs
#73Earlier quoted context omitted.
Do you automate your process somehow? I'm trying to develop a strategy where if I open my laptop after a certain time it backups in the background. My go to for this kind of thing was hammerspoon but it won't execute the shell task in the background thread and completely locks my machine. I guess I could cron it?
I've made systemd profiles and timers to do restic backups. It's really clean and works well. Also much better testable than cron. It will most likely also work for more complex scenarios, like you're mentioning.
Any chance you could clue me in on the scripts? Or point me to some resources to learn systemd? I've really disliked cron since having to use it for some auto push CI/CD stuff.
Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs
#74Backblaze is also the founding member of Bandwidth Alliance, meaning getting those B2 via Cloudflare is essentially free. So you are only paying for Storage. ( Correct me if I am on this one ) I wonder why doesn't ALL non HyperScale Cloud Vendors, like Linode and DO provide one click third party backup to B2. You should always store offsite backup somewhere. And B2 is perfect.
Does AWS (and GCP and Azure) not do the exact same thing when transferring data out via Cloudfront (or their respective CDN)? Is Backblaze really special in that regard?
Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs
#75There is no mention of the durability guarantees that s3 has.
Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs
#76Huh, I thought it already had this! Must have mixed it up with a different object storage service (maybe DigitalOcean?). I've been using B2 for backup storage for some personal projects. It doesn't necessarily do anything "better" than S3 from what I've seen, but never having to log into AWS's dashboard is a reward enough on its own. They do have a command-line client that's a quick PIP install, so you can do somethi…
I really wish the B2 client support uploading file from Unix pipe. It would be nice to be able to archive a huge directory into a tar.bz archive and directly pipe the result into the B2 client without having to save the archive into disk first.
Currently I have to save the tar.bz archive to disk first before uploading to balance. Took several hours to do so (huge spinning disks array, not as fast as ssd), while uploading to B2 is blazing fast. Saving the archive to ramdrive essentially solved this, but as the data grows I don't have enough memory to spare anymore for a ram drive that can fit the whole archive.
Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs
#77This is great. Their current API requires you to identify a unique host to send data to, so you’re constantly performing a metric ton of DNS queries. Until I white listed the base domain it was the #1 client of my Pihole installation by multiple orders of magnitude.
The DNS resolver library of your client is allowed to cache the IP address for a given hostname for up to TTL. If it does so, the cost should be negligible.
[deleted]
Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs
#78Earlier quoted context omitted.
The Cloudflare ToS explicitly exclude that use-case. 2.8 Limitation on Serving Non-HTML Content The Service is offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as a part of a Paid Service purchased by you, you agree to use the Service solely for the purpose of serving web pages as viewed through a web browser or other functionally equivalent applications and render…
From what I can tell that's more about their caching service. It mentions exceptions for hosting from a paid service and correct me if I'm wrong but wouldn't Backblaze be that paid service?
They are talking about their own paid service, obviously. I don't think they care whether you are paying for someone else's product or not.
Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs
#79Backblaze is also the founding member of Bandwidth Alliance, meaning getting those B2 via Cloudflare is essentially free. So you are only paying for Storage. ( Correct me if I am on this one ) I wonder why doesn't ALL non HyperScale Cloud Vendors, like Linode and DO provide one click third party backup to B2. You should always store offsite backup somewhere. And B2 is perfect.
Does AWS (and GCP and Azure) not do the exact same thing when transferring data out via Cloudfront (or their respective CDN)? Is Backblaze really special in that regard?
Cloudfront costs a non-trivial amount of money per GB: https://aws.amazon.com/cloudfront/pricing/
Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs
#80Earlier quoted context omitted.
The Cloudflare ToS explicitly exclude that use-case. 2.8 Limitation on Serving Non-HTML Content The Service is offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as a part of a Paid Service purchased by you, you agree to use the Service solely for the purpose of serving web pages as viewed through a web browser or other functionally equivalent applications and render…
From what I can tell that's more about their caching service. It mentions exceptions for hosting from a paid service and correct me if I'm wrong but wouldn't Backblaze be that paid service?
Perhaps they are talking about a paid service that they offer,like Cloudflare Stream https://www.cloudflare.com/products/cloudflare-stream/