LLM Microserving: a new RISC-style approach to design LLM serving API
1–2 of 2 posts
Re: LLM Microserving: a new RISC-style approach to design LLM serving API
#2Scale LLM serving with programmable cross-engine serving patterns, all in a few lines of Python