Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

backblaze.com

Post reply on HN
Print view View original

221–230 of 293 posts

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#221

Post by Ineentho »

neurostimulant wrote: post #191
jorams wrote: post #107

Earlier quoted context omitted.

Can you use process substitution? b2 upload_file bucket The argument the command sees will be something like "/dev/fd/42", and the shell will provide the output of tar through that file.

Does process substitution actually wrote the content to disk first or not? The information on internet I found seem to be conflicting on this. If it's actually writing the data into disk first, then it's probably won't solve my problem (limited disk i/o). Afaik writing to pipe won't result in saving the data to disk temporarily. I guess the only way to know is to try it out on my system and see how it performs.

If process substitution doesn't work, shouldn't /dev/stdin work? I haven't tried it, but as long as b2 doesn't try to check the file size before uploading I don't see why it wouldn't work: b2 upload_file bucket /dev/stdin < file

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#223

Post by chx »

jcampbell1 wrote: post #157
derefr wrote: post #122

Earlier quoted context omitted.

> So you would need to chunk up your videos to get free bandwidth. Having a functional video player on your site or in your app (e.g. one where you can skip to arbitrary times without requiring the video be buffered up to that point; or where a video can be "resumed" from the middle if you leave it and come back) already requires that you use MPEG-DASH or HLS; which in turn implies/necessitates pre-chunking, no? Is t…

You don’t need DASH or HLS to seek to an arbitrary point quickly. The mp4 container format has an index which stores playhead time -> byte. This index is either at the end of the file or beginning, and tools like qt-faststart move the index to the beginning, which makes videos start much quicker when serving over http. Browsers will use the index to issue range get requests an be able to seek just fine. Serving a 400…

ffmpeg.exe -i input.mp4 -c copy -movflags faststart -y output.mp4

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#224

Post by sundbry »

rb808 wrote: post #71

Is there a cheap s3 compatible service that is less reliable? I dont want to pay for redudancy Eg its my backups I can handle a 3% chance that my data is lost as long as I find out about it.

If you want super cheap, use DreamObjects by DreamHost.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#225

Post by cbo100 »

sparrc wrote: post #222

Does this mean I can use awscli to interact with b2 by specifying some backblaze server with --endpoint-url? What is the endpoint I would use?

When you create the bucket it shows a url along with the keys.

Not sure how unique that URL is, looking at the structure it could depend what data centre your bucket gets created in.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#226

Post by gaul »

DavideNL wrote: post #137
jedberg wrote: post #119

This is huge because it means you can use things like S3 Fuse to mount your storage. Which means you can use it to extend your local disk, or run your own backups, or whatever. Amusingly the price to store 1.2TB of data is the same as the cost of their backup plan, so if your disk is smaller than that, you could save a few bucks running your own backups. Until you have to restore (from what I can tell restores are fr…

> Which means you can use it to .... or run your own backups You could, but if i read correctly (s3fs-fuse limitations): "random writes or appends to files require rewriting the entire file". So changing 1 bit of a 10GB file, means re-uploading 10GB. https://github.com/s3fs-fuse/s3fs-fuse#limitations

This changed in 1.86 and I updated the README as follows:

> random writes or appends to files require rewriting the entire object, optimized with multi-part upload copy

Now changing one bit means re-uploading 5 MB, the minimum S3 part size.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#227

Post by [deleted] »

anotheracct_ wrote: post #163
atYevP wrote: post #58

Earlier quoted context omitted.

Yev from Backblaze here -> That's correct. We're a founding member of the Bandwidth Alliance and you can find more info about that on our Blog ( https://www.backblaze.com/blog/backblaze-and-cloudflare-part... ) or FAQ ( https://help.backblaze.com/hc/en-us/articles/217666928-Using... ). May the 4th be with you ;-)

Isn't this incompatible with Net Neutrality? I think Cloudflare already was against net neutrality, but people who believe in that principle might need to avoid Backblaze as well if that is the case.

[deleted]

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#228

Post by Dylan16807 »

derefr wrote: post #150
Dylan16807 wrote: post #147

Earlier quoted context omitted.

> Having a functional video player on your site or in your app (e.g. one where you can skip to arbitrary times without requiring the video be buffered up to that point; or where a video can be "resumed" from the middle if you leave it and come back) already requires that you use MPEG-DASH or HLS; which in turn implies/necessitates pre-chunking, no? Browsers are smart. They only buffer a few megabytes at a time and ca…

That requires the video to be encoded in a way where you can just start reading the stream from any random byte offset, and everything will still work. Video files are not usually encoded this way (any more.) Resume an MP4 or MKV video half-way through, without reading the TOC-ish stuff from the first chunk, and you'll get garbage that maybe resyncs after 20 seconds. It's totally possible to "encode for streaming", b…

The browser just reads the TOC-ish stuff from the first chunk. Trust me, it works. I regularly load plain old multi-hundred-megabyte mp4s in my browser, off a web server, and skip around without problems. The default keyframe interval from x264 is fine. You don't have to do any horrible things to the encoding, you just have to start loading a few seconds before the seek point. Which the browser does automatically.

Do browsers even support embedded subtitles?

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#229

Post by Dylan16807 »

sp332 wrote: post #203
anderspitman wrote: post #194

Earlier quoted context omitted.

That's odd. I can't recall ever having a problem with browsers playing mp4s in a vanilla tag, as long as I encode them in the main h264 profile, AAC audio, and MOOV atom at the front (see [0] for ffmpeg command). Obviously the server has to support Range: byte requests. My impression is DASH/HLS are mostly useful for adjusting bitrate on the fly. [0]: https://superuser.com/a/438471/402047

When the parent talks about jumping to random byte offsets, they mean you don't have the first part of the file at all. You just have an arbitrary 512-MB chunk out of the middle.

But they were claiming that a single monolithic file would break too, which is not the case. The browser does a range request to get the first part, then a range request to get the part you're playing, and it works.

Re: Backblaze B2 Cloud Storage Now Has S3 Compatible APIs

#230

Post by anotheracct_ »

NikolaeVarius wrote: post #218
anotheracct_ wrote: post #193

Earlier quoted context omitted.

So, Backblaze is not-net-neutral.

You are ignoring the responses to your original point. This is not relevant to net neutrality.

I think I addressed that comment, actually!
Post reply on HN