Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

backblaze.com

Post reply on HN
Print view View original

91–100 of 293 posts

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#92

Post by drenvuk »

atYevP wrote: post #58
ksec wrote: post #38

Backblaze is also the founding member of Bandwidth Alliance, meaning getting those B2 via Cloudflare is essentially free. So you are only paying for Storage. ( Correct me if I am on this one ) I wonder why doesn't ALL non HyperScale Cloud Vendors, like Linode and DO provide one click third party backup to B2. You should always store offsite backup somewhere. And B2 is perfect.

Yev from Backblaze here -> That's correct. We're a founding member of the Bandwidth Alliance and you can find more info about that on our Blog ( https://www.backblaze.com/blog/backblaze-and-cloudflare-part... ) or FAQ ( https://help.backblaze.com/hc/en-us/articles/217666928-Using... ). May the 4th be with you ;-)

Hey I've been wondering this but are the transaction costs also waived or is it just the bandwidth fees? I feel like if both are gone it'd be hard to make a profit...

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#93

Post by willcodeforfoo »

This is great news... there are lots more good clients for S3 than B2, and implementing one is less than trivial because of some special considerations B2 had in the beginning (namely: uploading directly to a pod.)

I see this isn't available for old buckets, is there a straightforward way to duplicate a bucket to make it compatible or do you have to use something like rclone?

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#94

Post by ac29 »

qmmmur wrote: post #73
Fritsdehacker wrote: post #46

Earlier quoted context omitted.

I've made systemd profiles and timers to do restic backups. It's really clean and works well. Also much better testable than cron. It will most likely also work for more complex scenarios, like you're mentioning.

Any chance you could clue me in on the scripts? Or point me to some resources to learn systemd? I've really disliked cron since having to use it for some auto push CI/CD stuff.

I just set this up last week, and it works well:

https://fedoramagazine.org/automate-backups-with-restic-and-...

edit: one gotcha is that using "user" systemd units, they will only run when the user is logged in. So, for a personal device like a laptop, this is fine, but for a server, it might not do what you're thinking. For the server use case, you probably want to enable linger for that utility user, so that units will run even with no active logged in sessions:

# loginctl enable-linger username

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#95

Post by willcodeforfoo »

heipei wrote: post #61

Now that we're talking about B2, has or is anyone using them for latency-sensitive small-file object storage? I'm about to take the plunge and set up benchmarks, my use-case is that I want to store and serve ~ 500k small files (30b-1MB) per day to website visitors. So far B2 support has told me that it shouldn't be a problem, and early benchmarking indicates the same, just curious if anyone had stories from the trenc…

We use B2 to store images on Vintage Aerial (https://vintageaerial.com), both high res scans and all kinds of thumbnail sizes.

It is... a little slower than I'd like but with Cloudfront in front it has been manageable. I love tips from Backblaze on how to increase performance there beyond caching to CF.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#96

Post by js4ever »

hemancuso wrote: post #50

As a developer that supports B2 (I write ExpanDrive) I think it’s great that they are moving on from an API that doesn’t expose any extra value. That being said, I wish B2 performance was better. Throughput is dramatically slower than S3.

B2 pricing is sooo attractive ... But I have been stopped by the much lower performance compared to S3 ... So unfortunately I can confirm this point :(

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#97

Post by l3s2d »

qmmmur wrote: post #37
l3s2d wrote: post #25

Earlier quoted context omitted.

I would recommend restic with B2. Really simple to set up. I use it for nightly homedir backups for multiple machines. Currently at ~400GB and it's costing me < $3/month.

Do you automate your process somehow? I'm trying to develop a strategy where if I open my laptop after a certain time it backups in the background. My go to for this kind of thing was hammerspoon but it won't execute the shell task in the background thread and completely locks my machine. I guess I could cron it?

I use a systemd user service. It sounds like you're on OS X, so you won't have systemd. Cron should work.

Here are my systemd files for reference.

Service:

  [Unit]
  Wants=network-online.target
  After=network-online.target

  [Service]
  Type=simple
  ExecStart=/bin/bash -c "/usr/bin/restic unlock && /usr/bin/restic backup --verbose --one-file-system --exclude-caches --exclude=$XDG_CACHE_HOME %h && /usr/bin/restic forget --keep-last 5 --prune && /usr/bin/restic cache --cleanup --max-age 30"

  [Install]
  WantedBy=default.target
Environment (fill these in with your keys):

  [Service]
  Environment="B2_ACCOUNT_ID="
  Environment="B2_ACCOUNT_KEY="
  Environment="RESTIC_REPOSITORY="
  Environment="RESTIC_PASSWORD="
Timer:

  [Timer]
  OnCalendar=*-*-* 00:30:00
  RandomizedDelaySec=20min

  [Install]
  WantedBy=timers.target

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#98

Post by heipei »

willcodeforfoo wrote: post #95
heipei wrote: post #61

Now that we're talking about B2, has or is anyone using them for latency-sensitive small-file object storage? I'm about to take the plunge and set up benchmarks, my use-case is that I want to store and serve ~ 500k small files (30b-1MB) per day to website visitors. So far B2 support has told me that it shouldn't be a problem, and early benchmarking indicates the same, just curious if anyone had stories from the trenc…

We use B2 to store images on Vintage Aerial ( https://vintageaerial.com ), both high res scans and all kinds of thumbnail sizes. It is... a little slower than I'd like but with Cloudfront in front it has been manageable. I love tips from Backblaze on how to increase performance there beyond caching to CF.

Can you go into detail what you mean by "slower than I'd like"? Are you talking about TTFB (Time-To-First-Byte) or sustained read or concurrent read performance? Are you using the API or the HTTP endpoints from a public bucket?

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#100

Post by budmang »

rochester46 wrote: post #74
ksec wrote: post #38

Backblaze is also the founding member of Bandwidth Alliance, meaning getting those B2 via Cloudflare is essentially free. So you are only paying for Storage. ( Correct me if I am on this one ) I wonder why doesn't ALL non HyperScale Cloud Vendors, like Linode and DO provide one click third party backup to B2. You should always store offsite backup somewhere. And B2 is perfect.

Does AWS (and GCP and Azure) not do the exact same thing when transferring data out via Cloudfront (or their respective CDN)? Is Backblaze really special in that regard?

(backblaze ceo here) AWS is free to Cloudfront because that's an AWS service. AWS/GCP/Azure are not free (and quite expensive) to Cloudflare. Backblaze is free to Cloudflare.
Post reply on HN