I'm working on a Python package for this: https://github.com/qagentur/texttunnel It's a wrapper for OpenAI's Python sample script plus adjacent functionality like cost estimation and binpacking multiple inputs into one request.
Will check it out!
Ask HN: Batch Processing with OpenAI
11–12 of 12 posts
Re: Ask HN: Batch Processing with OpenAI
#12You could set up a collection in memory/SQL with a Status + UpdatedUtc column and poll the collection for incomplete items each loop until all are in the desired state. Your state machine could be as simple as: New, Processing, Failed, Succeeded. Outer loop will query the collection every ~second for items that are New or Failed and retry them. Items that are stuck Processing for more than X seconds should be forced…
not op, but this is a neat idea, thanks.