Static vs. Dynamic vs. Continuous Batching in LLM Inference

machinelearningmastery.com