For anyone curious, here is an early prototype of this tech in action:
Would love to hear your thoughts on how we can make this most useful for you!
Ask HN: Would you use a shared GPU cloud tier?
11–20 of 20 posts
Re: Ask HN: Would you use a shared GPU cloud tier?
#12Sounds like an extremely complex technical problem. I also suggest to look at the use cases when this is needed. One of the problems is that loading weights into the GPU will be so slow that it will be really hard to share the GPU between different processes - causing long time to offload and load. Would love to learn more about what you do.
Totally right on all fronts. This has been tough to build and we are still searching for the best use cases. If you are down to chat my email is in my profile, any insights would help us a ton!
Re: Ask HN: Would you use a shared GPU cloud tier?
#13Yes, I have wanted something like this for a while. I try to avoid using gpus where possible because of the expense, and the ephemeral nature of my use.
Interesting to hear. What kind of workloads are you running?
Re: Ask HN: Would you use a shared GPU cloud tier?
#14For anyone curious, here is an early prototype of this tech in action: https://imgur.com/a/2qPN4ru Would love to hear your thoughts on how we can make this most useful for you!
woah that's really cool! can't wait for the show HN
Re: Ask HN: Would you use a shared GPU cloud tier?
#15Yes, I have wanted something like this for a while. I try to avoid using gpus where possible because of the expense, and the ephemeral nature of my use.
Interesting to hear. What kind of workloads are you running?
Various forms of content analysis. It is mostly traditional heuristics with ml models sprinkled in. We don't run inference on every request, but it would be great to access a dynamicly provisioned gpu for a single request (kind of like its another serverless container in the system).
Re: Ask HN: Would you use a shared GPU cloud tier?
#16Earlier quoted context omitted.
Interesting to hear. What kind of workloads are you running?
Various forms of content analysis. It is mostly traditional heuristics with ml models sprinkled in. We don't run inference on every request, but it would be great to access a dynamicly provisioned gpu for a single request (kind of like its another serverless container in the system).
That's fascinating to hear and I think it would work really well with what we do.
What I am picturing is that you could run the whole workflow including traditional heuristics in a CPU instance, which would connect to a GPU on-demand.
If you are interested would love for you to try this. We're running a (very unprofitable) beta with a T4 instance + a CPU-only instance for $10/month for those who are willing to help us test this with production workloads. If you'd be interested would love to chat at carl (at) thundercompute (dot) com.
Re: Ask HN: Would you use a shared GPU cloud tier?
#17Depending on the price difference from a standard GPU instance, absolutely.
Hey Einsteinx2, we're unsure where the pricing will end up at scale, however we are running a beta with a T4 instance + a CPU-only instance for $10/month, so can guarantee a pretty crazy deal for now :)
If you'd be interested would love to chat more at carl (at) thundercompute (dot) com.
Re: Ask HN: Would you use a shared GPU cloud tier?
#18Yes, I would use it if the price is affordable.
We're running a beta for $10/month for a T4, which is definitely affordable, but we are running at a steep loss for now to test our system at scale. If you're down to be one of the first few users email me at carl (at) thundercompute (dot) com!
Re: Ask HN: Would you use a shared GPU cloud tier?
#19For anyone curious, here is an early prototype of this tech in action: https://imgur.com/a/2qPN4ru Would love to hear your thoughts on how we can make this most useful for you!
woah that's really cool! can't wait for the show HN
Appreciate it! We're reaching out on a smaller scale for now to manage how many signups we get to not break our systems. Happy to share a demo vid and/or more details at carl (at) thundercompute (dot) com.
Re: Ask HN: Would you use a shared GPU cloud tier?
#20Sounds like an extremely complex technical problem. I also suggest to look at the use cases when this is needed. One of the problems is that loading weights into the GPU will be so slow that it will be really hard to share the GPU between different processes - causing long time to offload and load. Would love to learn more about what you do.
Hi Cheptsov, know you mentioned wanting to learn more. We have some demo material that we can share if you're interested. Would love to chat at carl (at) thundercompute (dot) com.