On incomplete HTTP reads and the requests library in Python

Post by cessor » Mon, Apr 23, 2018, 12:59 PM UTC

crdoconnor wrote: post #30

>Secondly, if we throw an exception we irrevocably destroy the data we read. It becomes impossible to access. This means that situations where the user might want to ‘muddle through’, taking as much of the data as they were able to read and keeping hold of it, becomes a little bit harder. This is an awful justification. They should do this by raising an exception and attaching the response data to it, so the user who…

I fully agree with you.

Altough I really like requests, this is a violation of one of the PEP 20 heuristics:

Errors should never pass silently. Unless explicitly silenced.

https://www.python.org/dev/peps/pep-0020/

An exception would be an elegant way to handle the problem and would be able to retain the incomplete data to be handeled. It's exactly what they are for. The exception could be the default to prevent surprises, possible deactivated with a flag.

Post by cryptonector » Mon, Apr 23, 2018, 1:51 PM UTC

greglindahl wrote: post #12

cryptonector wrote: post #2

I've written an HTTP service that offers "tail -f" as a service. When you GET a resource with a Range: bytes=0- (or some other starting offset with unspecified end offset) you get chunked transfer-encoding that doesn't terminate until the file is unlinked or renamed out of the way. This is incredibly handy, both for the usual things one might want to tail -f (log files), and as a cheap-but-very-functional pub/sub sys…

You could consider using websockets, which are intended to be used the way you're trying to use chunking.

I'm aware. But this is easier to use. For example, here's how you tail some log file: curl -H 'Range: bytes=0-' https://foo.example/bar.log.

Re: On incomplete HTTP reads and the requests library in Python

Re: On incomplete HTTP reads and the requests library in Python