Live data from Hacker News

Cutting LLM Batch Inference Time in Half: Dynamic Prefix Bucketing at Scale

Post reply on HN

Print view View original

1–2 of 2 posts

Cutting LLM Batch Inference Time in Half: Dynamic Prefix Bucketing at Scale

#1

Post by ykev » Tue, Nov 04, 2025, 5:16 PM UTC

Cutting LLM Batch Inference Time in Half: Dynamic Prefix Bucketing at Scale
daft.ai

Re: Cutting LLM Batch Inference Time in Half: Dynamic Prefix Bucketing at Scale

#2

Post by sammysidhu » Tue, Nov 04, 2025, 5:21 PM UTC

Part of the Daft team here! Happy to answer any questions

Post reply on HN