Can you explain the benefits over something like openrouter?
24/7 LLM for $10/month.
Show HN: sllm – Split a GPU node with other developers, unlimited tokens
41–50 of 117 posts
Re: Show HN: sllm – Split a GPU node with other developers, unlimited tokens
#42Earlier quoted context omitted.
vLLM handles GPU scheduling, not sllm. The model weights stay resident in VRAM permanently so there's no loading/unloading per request. vLLM uses continuous batching, so incoming requests are dynamically added to the running batch every decode step and the GPU is always working on multiple requests simultaneously. There is no "load to VRAM and run" per request; it's more like joining an already-running batch. TTFT is…
> The model weights stay resident in VRAM permanently so there's no loading/unloading per request. Yes, I was thinking about context buffers, which I assume are not small in large models. That has to be loaded into VRAM, right? If I keep sending large context buffers, will that hog the batches?
[deleted]
Re: Show HN: sllm – Split a GPU node with other developers, unlimited tokens
#43> How does billing work? > When you join a cohort, your card is saved but not charged until the cohort fills. Stripe holds your card information — we never store it. Once the cohort fills, you are charged and receive an API key for the duration of the cohort. Have any cohorts filled yet? I’m interested in joining one, but only if it’s reasonable to assume that the cohort will be full within the next 7 days or so. (Es…
No cohorts have been filled yet. We're still early. We are seeing reservations pick up quickly, but I'd be able to give you a more concrete estimate of fill velocity after about a week.
That said, we're planning to add a 7-day window: if a cohort doesn't fill within 7 days of your reservation, it cancels automatically and your card is released. We don't want anyone's payment method sitting in limbo indefinitely.
Re: Show HN: sllm – Split a GPU node with other developers, unlimited tokens
#44Earlier quoted context omitted.
vLLM handles GPU scheduling, not sllm. The model weights stay resident in VRAM permanently so there's no loading/unloading per request. vLLM uses continuous batching, so incoming requests are dynamically added to the running batch every decode step and the GPU is always working on multiple requests simultaneously. There is no "load to VRAM and run" per request; it's more like joining an already-running batch. TTFT is…
> The model weights stay resident in VRAM permanently so there's no loading/unloading per request. Yes, I was thinking about context buffers, which I assume are not small in large models. That has to be loaded into VRAM, right? If I keep sending large context buffers, will that hog the batches?
Not if you are the only one. We have rate limits to prevent this in case, idk, you share your key with 1000 people lol.
Re: Show HN: sllm – Split a GPU node with other developers, unlimited tokens
#45This is great, thanks!
I personally would like something like this but with "regular" GPU access. Some people still use them for something other than LLMs ^^.
Re: Show HN: sllm – Split a GPU node with other developers, unlimited tokens
#46> Running DeepSeek V3 (685B) requires 8×H100 GPUs which is about $14k/month. Most developers only need 15-25 tok/s.
> deepseek-v3.2-685b, $40/mo/slot for ~20 tok/s, 465 slots total
> 465 users × 20 tok/s = 9,300 tok/s needed
> The node peaks at ~3,000 tok/s total. So at full capacity they can really only serve:
> 3,000 ÷ 20 = 150 concurrent users at 20 tok/s
> That's only 32% of the cohort being active simultaneously.
Re: Show HN: sllm – Split a GPU node with other developers, unlimited tokens
#47Didn't make sense to launch multiple 10 and 40 bucks subscriptions right at the start, because now they are competing with each other.
Also mobile version is a bit broken, but good idea and good luck!
Re: Show HN: sllm – Split a GPU node with other developers, unlimited tokens
#48> Running DeepSeek V3 (685B) requires 8×H100 GPUs which is about $14k/month. Most developers only need 15-25 tok/s. > deepseek-v3.2-685b, $40/mo/slot for ~20 tok/s, 465 slots total > 465 users × 20 tok/s = 9,300 tok/s needed > The node peaks at ~3,000 tok/s total. So at full capacity they can really only serve: > 3,000 ÷ 20 = 150 concurrent users at 20 tok/s > That's only 32% of the cohort being active simultaneously…
People work 8 hours a day presumably, I guess they are banking on this idea
Re: Show HN: sllm – Split a GPU node with other developers, unlimited tokens
#49Didn't make sense to launch multiple 10 and 40 bucks subscriptions right at the start, because now they are competing with each other. Also mobile version is a bit broken, but good idea and good luck!
I'm feeling it Mr. Crabs.
Re: Show HN: sllm – Split a GPU node with other developers, unlimited tokens
#50Do you own the GPUs or are you multiplexing on a 3rd party GPU cloud?
Multiplexing on a GPU cloud.