Viewing profile — Unheard3610

Follow on, great question (separately from John) can it run on Vulkan? He was saying he has a hard time finding stuff that doesn’t only run on CUDA or ROCm and sees this as a huge …

Yeah! Or maybe the GPU rental and lab market? (At least those that aren’t already fully hyperscaler captured?) like we were chatting about John?

but wait, why should I do this for my first home grown orchestration instead of something else? Like, if I want to set up a local LLM running on my old laptop for some kind of RAG …