Live data from Hacker News

Open Protocol for Resumable File Uploads

tus.io

11–20 of 45 posts

Re: Open Protocol for Resumable File Uploads

#11

The HTML5 FileAPI has been around for a few years now yet a lot of sites don't support resumable uploads. I know it adds a bunch of complexity server side as you have to restitch those pieces together but it makes for a good user experience.

I hope with a client like https://uppy.io and a server like tusd, it’s much more manageable these days. Less boilerplate writing and more battle tested components for sure.

Re: Open Protocol for Resumable File Uploads

#12

What’s wrong with HTTP PUT with Content-Range?

  > An origin server that allows PUT on a given target resource MUST send
  > a 400 (Bad Request) response to a PUT request that contains a
  > Content-Range header field (Section 4.2 of [RFC7233]), 
https://tools.ietf.org/html/rfc7231#section-4.3.4

Re: Open Protocol for Resumable File Uploads

#13
post #12

What’s wrong with HTTP PUT with Content-Range?

> An origin server that allows PUT on a given target resource MUST send > a 400 (Bad Request) response to a PUT request that contains a > Content-Range header field (Section 4.2 of [RFC7233]), https://tools.ietf.org/html/rfc7231#section-4.3.4

Maybe that should be fixed, then. HTTP PUT with the range specified seem to me it would be sensible.

Re: Open Protocol for Resumable File Uploads

#14
post #12

Earlier quoted context omitted.

> An origin server that allows PUT on a given target resource MUST send > a 400 (Bad Request) response to a PUT request that contains a > Content-Range header field (Section 4.2 of [RFC7233]), https://tools.ietf.org/html/rfc7231#section-4.3.4

Maybe that should be fixed, then. HTTP PUT with the range specified seem to me it would be sensible.

Responding with 400 Bad Request is actually something that was added after some servers allowed Content-Range on PUT and others didn't.

It was never standard, but the end-result was that some clients assumed PUT + Content-Range would work, which meant that some servers would apply the change while others would ignore the header and overwrite the entire resource with the chunk.

There's no sane way to add support for this header and make older servers behave correctly, so now we have better facilities for this.

The standard way is to use PATCH + a mimetype that describes the update + perhaps using Accept-Patch to find out what formats are available. It's extremely doubtful that Content-Range for PUT will ever be standard. If there's going to be a future standard, it's likely PATCH based.

It could be possible with PUT and a new 'Expect' header, but not sure if that gives any advantages now over PATCH.

Re: Open Protocol for Resumable File Uploads

#15
post #6

rsync? Yes I know this is mainly for browsers.

Yes for browsers it’s cheaper to build upon http, and it let’s you move through airport/hotel/corporate firewalls without problems.

Tus is also used in datacenters for high throughput & reliable transmissions. Probably in most cases rsync is a sensible choice, but sometimes maybe you already have tus, http based auth, loadbalancing, etc in place that you want to leverage, or maybe you want to avoid exchanging ssh secrets

Re: Open Protocol for Resumable File Uploads

#17
If I read the spec correctly, PATCH method is actually more of APPEND, no?

It would seem logical and practical to allow PATCH to modify any part of a resource that is already present on the server and/or to extend it by appending. This would also make the whole thing useful beyond resuming of interrupted uploads, e.g. to allow for rsync-style updating of existing files.

Re: Open Protocol for Resumable File Uploads

#20

S3 Multi-Part Upload API can be used to chunk an object into smaller parts, which can succeed or fail independently. https://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview....

Yes that is very helpful. Our s3 storage backend for tusd uses it, and our https://uppy.io file uploader does too, usable directly from the browser (so you can choose to not use tus at all with it). S3 resumable uploads do come with a few limitations that make some people still choose tus tho:

* chunks need to be >5MB which can be problematic on flaky/poor connections (rural areas, tunnels, clubs/basements, people on the move switching connections all the time)

* your s3 bucket needs to allow write by the world, or you need to deploy signature authentication

* there’s an s3 vendor lock-in some might worry about

* not an open protocol, no chance of advancing it with the community

That said, that still leaves a large audience for direct s3 resumable uploads and I’m thankful aws offers it!

Post reply on HN