Earlier quoted context omitted.
Yes, that is that I mean. All the modern "ChatGPT" style API's use this, so if you're building anything that invokes them you can choose between buffering the entire response and modifying/relaying it once complete, or building up a toolchain of streaming-capable utilities. Of all the http-client applications I tested (Curl, Postman, Insomnia, Bruno), somewhat hilariously Curl has the best support. It will output all…
What if instead of JSON, i.e., strings of unknown length, used something more like netstrings. https://cr.yp.to/proto/netstrings.txt Personally I use a lame but effective simple 85.9 KiB static binary filter, a small C program, that removes the chunk sizes so the response is ready for use by other programs, e.g., in a pipe. Buffer is set at 8 KiB. Is there a way to experiment with one of these streaming JSON GPT APIs…
I don't know a public API that returns JSON slowly, but you could simulate it by just taking a JSON string, splitting it into 3-5 char chunks, and sending each of those in a `Transfer-Encoding: Chunked` response at ~100ms intervals.
Actually, now that I look at the underlying mechanism behind `Transfer-Encoding: Chunked`, it looks like it's already basically the same as the netstrings. What I'm referring to is the (variable length) contents of the netstring/chunk being sequential slices of a JSON object.