Continuous batching enables 23x throughput in LLM inference #1 Post by richardliaw » Fri, Jun 23, 2023, 6:30 PM UTC Continuous batching enables 23x throughput in LLM inferenceanyscale.com