Shameless plug for my multipart crate: https://github.com/cetra3/mpart-async which I've been using happily in production for a long time now
Introduction to HTTP Multipart
11–18 of 18 posts
Re: Introduction to HTTP Multipart
#12Does HTTP/2 have support for parallel uploads? As an end-user if I happen to select 1 huge file first, followed by a bunch of tiny files, having to wait for the entire huge file to be streamed to the server before it even knows about any of the others seems less than ideal. It also seems odd there's no way to provide a hint to the server about individual file-sizes up-front (only Content-Length which is your upper bo…
You can technically add a Content-Length header for each part. It's not forbidden by the RFC, but nor is it common. It caused problems (https://github.com/square/okhttp/issues/2138) for OkHttp, and they eventually removed it. Might be fine for internal-only use, though.
Boundaries are a lot like UUIDs, and rely on the same logic. When generating random data, once you have enough bits, the odds are against that sequence of bits ever having been generated before in the universe.
Re: Introduction to HTTP Multipart
#13> You can gzip the entire Multipart response, but you cannot pick and choose compression for particular parts.
Sure you can. I designed a system where the client uploads a multipart-form that's three parts, the first part is JSON ("meta") and the next two parts are gzipped blobs ("raw.gz" and "log.gz"). The server reads the first part which is metadata that tells it how to handle the next two parts.
I happen to be using Falcon and streaming-form-data on the server side.
Re: Introduction to HTTP Multipart
#14The article talks about multipart/form-data in particular. Another thing one might run across is multipart/x-mixed-replace. I wrote a crate for that. [1] I didn't see a spec for it, but someone since pointed out to me that it's probably identical to multipart/mixed, and now seeing an example in the multer README it clicks that I should have looked at RFC 2046 section 5.1.1, [2] which says this: > This section defines…
Its specified in html5 https://html.spec.whatwg.org/multipage/document-lifecycle.ht...
However chrome killed support except for images https://bugs.chromium.org/p/chromium/issues/detail?id=249132
Re: Introduction to HTTP Multipart
#15Does HTTP/2 have support for parallel uploads? As an end-user if I happen to select 1 huge file first, followed by a bunch of tiny files, having to wait for the entire huge file to be streamed to the server before it even knows about any of the others seems less than ideal. It also seems odd there's no way to provide a hint to the server about individual file-sizes up-front (only Content-Length which is your upper bo…
Yes, HTTP/2 requests run on separate streams and will interleave chunks. They still have some head of line blocking however, since the TCP stream will not allow reading from another stream while waiting for another stream’s chunk to retransmit (for that you need HTTP/3).
HTTP/1 requests (uploads in this case) are also separate to some degree (though there are fairly stringent limits on connections per domain iirc which HTTP/2 resolves via the mentioned streams/multiplexing of connections).
The problem they have specifically would be that in a single request (form post for example) those uploads will be linear.
Solution really boils down to paralellizing the upload, using protocols/standards like https://tus.io/ or S3-compatible APIs to push the data up then syncronize with a record/document on the server.
Re: Introduction to HTTP Multipart
#16> You can gzip the entire Multipart response, but you cannot pick and choose compression for particular parts. Sure you can. I designed a system where the client uploads a multipart-form that's three parts, the first part is JSON ("meta") and the next two parts are gzipped blobs ("raw.gz" and "log.gz"). The server reads the first part which is metadata that tells it how to handle the next two parts. I happen to be us…
> I happen to be using Falcon
heh I'm always annoyed by this unfortunate name clash:
Re: Introduction to HTTP Multipart
#17Does HTTP/2 have support for parallel uploads? As an end-user if I happen to select 1 huge file first, followed by a bunch of tiny files, having to wait for the entire huge file to be streamed to the server before it even knows about any of the others seems less than ideal. It also seems odd there's no way to provide a hint to the server about individual file-sizes up-front (only Content-Length which is your upper bo…
You can technically add a Content-Length header for each part. It's not forbidden by the RFC, but nor is it common. It caused problems ( https://github.com/square/okhttp/issues/2138 ) for OkHttp, and they eventually removed it. Might be fine for internal-only use, though. Boundaries are a lot like UUIDs, and rely on the same logic. When generating random data, once you have enough bits, the odds are against that sequ…
EDIT: Was looking at the obsolete RFC. The current version, RFC 7578, actually forbids all part headers other than "Content-Type, Content-Disposition, and (in limited circumstances) Content-Transfer-Encoding".
Re: Introduction to HTTP Multipart
#18I wonder why we didn't use a framed representation rather than delimiters that have to be searched for (which isn't so simple). It makes writing a streaming MIME parser much harder. With a compulsory Content-Length things would be much easier. At least with multipart/form-data we get to avoid transfer encodings, which are also quite annoying to handle (especially as they can be nested, which is probably the worst asp…
Streaming. What if the part(s) are being generated on the fly, and don't exist to have a length? Requiring that the sender specify the length up front requires the length to be known. Maybe I'm (e.g.) tarring/zipping up a lot of data, and I don't want to hold it all in memory/write it to a temporary file first.