Reverse-engineering Nvidia's CUDA-checkpoint for faster cold starts

#1

Post by ilreb » Thu, Jul 09, 2026, 1:29 PM UTC

Reverse-engineering Nvidia's CUDA-checkpoint for faster cold starts
blog.doubleword.ai

Re: Reverse-engineering Nvidia's CUDA-checkpoint for faster cold starts

#2

Post by zoobab » Thu, Jul 09, 2026, 7:18 PM UTC

Would it be able to swap models on demand with this?

Re: Reverse-engineering Nvidia's CUDA-checkpoint for faster cold starts

#3

Post by fergus00 » Thu, Jul 09, 2026, 8:01 PM UTC

zoobab wrote: post #2

Would it be able to swap models on demand with this?

https://blog.doubleword.ai/fast-sglang-starts yeah, this is part of the idea. If you get cold starts down to seconds or subseconds, then you can run many models multiplexed on the same GPUs

Re: Reverse-engineering Nvidia's CUDA-checkpoint for faster cold starts

#4

Post by zoobab » Thu, Jul 09, 2026, 8:04 PM UTC

fergus00 wrote: post #3

zoobab wrote: post #2

Would it be able to swap models on demand with this?

https://blog.doubleword.ai/fast-sglang-starts yeah, this is part of the idea. If you get cold starts down to seconds or subseconds, then you can run many models multiplexed on the same GPUs

I have read a Linkedin post in french recently with a university lab (one big machine serving hundreds of users) mentioning that they could swap models on demand. Will try to find back the link.