Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

backblaze.com

Post reply on HN
Print view View original

81–90 of 293 posts

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#81

Post by celicaraptor »

Hamuko wrote: post #41
ksec wrote: post #38

Backblaze is also the founding member of Bandwidth Alliance, meaning getting those B2 via Cloudflare is essentially free. So you are only paying for Storage. ( Correct me if I am on this one ) I wonder why doesn't ALL non HyperScale Cloud Vendors, like Linode and DO provide one click third party backup to B2. You should always store offsite backup somewhere. And B2 is perfect.

How does that work? Can I just dumb two terabytes of video into Backblaze B2, setup a Cloudflare account and have people watch those videos with it costing me only $10 a month? Because that doesn't sound right.

This thread might help https://news.ycombinator.com/item?id=20790857

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#82

Post by anon102010 »

Hamuko wrote: post #41
ksec wrote: post #38

Backblaze is also the founding member of Bandwidth Alliance, meaning getting those B2 via Cloudflare is essentially free. So you are only paying for Storage. ( Correct me if I am on this one ) I wonder why doesn't ALL non HyperScale Cloud Vendors, like Linode and DO provide one click third party backup to B2. You should always store offsite backup somewhere. And B2 is perfect.

How does that work? Can I just dumb two terabytes of video into Backblaze B2, setup a Cloudflare account and have people watch those videos with it costing me only $10 a month? Because that doesn't sound right.

This is huge if true - folks keep on saying Cloudflare can replace cloudfront for free. If we could put 20TB of binary content for software distro / videos etc on B2, then stream by cloudflare I'd actually believe the claims.

Every time we've tried this with other "free" providers there always seems to be "fine print" when you start pushing huge bandwidth on these "unlimited" plans. It seriously is not worth the time at some point.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#83

Post by chx »

Hamuko wrote: post #41
ksec wrote: post #38

Backblaze is also the founding member of Bandwidth Alliance, meaning getting those B2 via Cloudflare is essentially free. So you are only paying for Storage. ( Correct me if I am on this one ) I wonder why doesn't ALL non HyperScale Cloud Vendors, like Linode and DO provide one click third party backup to B2. You should always store offsite backup somewhere. And B2 is perfect.

How does that work? Can I just dumb two terabytes of video into Backblaze B2, setup a Cloudflare account and have people watch those videos with it costing me only $10 a month? Because that doesn't sound right.

The way I understand, you can do this with images, eastdakota confirmed that here earlier https://news.ycombinator.com/item?id=20791660 not sure about videos.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#84

Post by ac29 »

sime2009 wrote: post #72
waffle_ss wrote: post #43

Earlier quoted context omitted.

That's a fair point but this is for my family photos and videos; in the event I die I'd rather the S3 bucket I hand over to my wife/kids in my last wishes look like a real filesystem rather than a bazillion blobs that require a special tool with a programmer's expertise to reassemble

May I ask which solution you ended up going for in this situation? (I'm trying to solve for myself too.)

rclone is a nice way to sync files to B2. It also has a mount option that quite literally mounts the cloud storage as a filesystem, though this requires Linux and a little bit of know how. However, in a pinch B2 has a serviceable web interface, and Backblaze will even ship you a drive of your files if you request it, so I think it would be pretty usable by just about anyone.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#85

Post by mgamache »

atYevP wrote: post #31
mgamache wrote: post #4

S3 is now the standard for cloud storage APIs? Not sure if that's good or bad. I guess competitors have to reduce switching costs.

Yev here -> It's not so much a standard, though S3 is generally the most often used suite of APIs. 100s of integrations exist with our B2 Native APIs ( https://www.backblaze.com/b2/integrations.html ), but a lot of folks only know how to write to S3 Compatible APIs and don't have the resources to write to multiple API suites, so this makes integration easier for them!

That's how something becomes a standard... You are just responding to market realities.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#86

Post by mikehearn »

heipei wrote: post #59
Hamuko wrote: post #41

Earlier quoted context omitted.

How does that work? Can I just dumb two terabytes of video into Backblaze B2, setup a Cloudflare account and have people watch those videos with it costing me only $10 a month? Because that doesn't sound right.

The Cloudflare ToS explicitly exclude that use-case. 2.8 Limitation on Serving Non-HTML Content The Service is offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as a part of a Paid Service purchased by you, you agree to use the Service solely for the purpose of serving web pages as viewed through a web browser or other functionally equivalent applications and render…

In a previous HN post [1] where someone wrote up how to use B2 as an image host, the Cloudflare CEO chimed in and addressed rule 2.8 specifically, saying if Cloudflare workers are used for URL prettifying and redirecting, a different ToS is applied and that use-case would be fine.

Does that mean your video use-case would also be fine? I have no idea. An HN comment from the CEO doesn't seem like it would hold up if Cloudflare suddenly shut down your free account.

I'd love for Cloudfront to officially clarify the limits of the Cloudfront/B2 alliance in terms of external traffic. The confounding issue here is that B2, as a storage service, is not really intended for "serving web pages and websites" — it's for larger files, binaries, etc. — and therefore any traffic from B2 going through Cloudfront is sort of de facto in violation of 2.8.

[1]: https://news.ycombinator.com/item?id=20790857

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#87

Post by jjeaff »

tambre wrote: post #13
nielsole wrote: post #3

Here is the reasoning why they didn't have "s3 compatibility" before: https://www.backblaze.com/blog/design-thinking-b2-apis-the-h...

>It requires Amazon to have a massive and expensive choke point in their network: load balancers. When a customer tries to upload to S3, she is given a single upload URL to use. For instance, s3.amazonaws.com/ . Now that Amazon has deprecated the single URL version and replaced it with region-specific URLs (e.g. s3.dualstack.us-east-1.amazonaws.com) and tooling has been mostly updated, this huge reason for not suppor…

Even though they are using region specific URLs, they would still have to load balance all of that traffic. Backblaze avoided this by having a 2 part request for a file. You would make a request to a centralized url and then that would return a url that connected you directly to a server that had that file.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#88

Post by rochester46 »

wrkronmiller wrote: post #79
rochester46 wrote: post #74

Earlier quoted context omitted.

Does AWS (and GCP and Azure) not do the exact same thing when transferring data out via Cloudfront (or their respective CDN)? Is Backblaze really special in that regard?

Cloudfront costs a non-trivial amount of money per GB: https://aws.amazon.com/cloudfront/pricing/

How does that compare to Cloudflare? Is Cloudflare completely free for CDN? I can't find a clear pricing page.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#89

Post by anon102010 »

solarkraft wrote: post #24
atYevP wrote: post #14

Earlier quoted context omitted.

Yev here -> Anyone can write to the B2 Native API or our S3 Compatible API, we have tons of integrations that do it, here's a list -> https://www.backblaze.com/b2/integrations.html

I don't think that's what gp meant. Why not use a standard protcol like SFTP?

what python blob storage libraries are using "SFTP"? How is that the "standard"?

I literally have NEVER seen SFTP being used for blob storage in any python project - is this a real thing somewhere?

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#90

Post by shanemhansen »

I'm curious what their load balancing layer looks like. There's alot of interesting options. (Disclaimer: I've worked in the CDN and the storage space in the past)

If their load balancer is smart enough it can call the dispatcher, and make use of something like https://zaiste.net/nginx_x_accel_header/ to figure out where to forward the request. Unfortunately this still requires uploads be proxied through the dispatcher.

You could get crazy and involve a CDN (akamai or cloudflare or fastly) that could do some smart logic, especially if you can emit your dispatcher as a lookup table that's updated frequently. I don't know what bandwidth costs would be for that though. Probably high.

It's an interesting problem space and I'd love to talk to these folks about it.

Post reply on HN