11–20 of 36 posts
> Max. model size: X GB
Do you really mean model size or should this also include the size of the intermediate tensors?
The full screen option on the YouTube video is turned off by the way, so it is impossible to read without leaving your website.
Overall, this offer looks quite competitive. Are you planning to offer your service in the EU in the future?
May I deploy multiple models in the same billing account?
> Guaranteed This sounds confusing to me. Surely it is possible to craft a neural network that takes longer to process? > Max. model size: X GB Do you really mean model size or should this also include the size of the intermediate tensors? The full screen option on the YouTube video is turned off by the way, so it is impossible to read without leaving your website. Overall, this offer looks quite competitive. Are you…
The model size is the zipped size of your model that is uploaded to Inferrd (either through the SDK or the website).
I'll fix the full screen problem right away, thank you for reporting.
We only have servers in the United States at the moment but are looking to have servers all around NA and EU very soon.
> Guaranteed This sounds confusing to me. Surely it is possible to craft a neural network that takes longer to process? > Max. model size: X GB Do you really mean model size or should this also include the size of the intermediate tensors? The full screen option on the YouTube video is turned off by the way, so it is impossible to read without leaving your website. Overall, this offer looks quite competitive. Are you…
The response time guaranteed is for a reasonably sized model. Bigger models (> 700 MB) will take a bit longer. The model size is the zipped size of your model that is uploaded to Inferrd (either through the SDK or the website). I'll fix the full screen problem right away, thank you for reporting. We only have servers in the United States at the moment but are looking to have servers all around NA and EU very soon.
Nice to hear!
> We only have servers in the United States at the moment but are looking to have servers all around NA and EU very soon.
Sorry, my question was not quite clear. What I actually wanted to know was more along the lines of being able to use your service in Europe legally. For example, I can not find a privacy policy or a way to get a GDPR data processing agreement.
CUDA 11.3.0
cuBLAS 11.5.1.101
cuDNN 8.2.0.41
NCCL 2.9.6
TensorRT 7.2.3.4
Triton Inference Server 2.9.0
I'm new to deploying to production inference so I'm not sure if those are easily portable across such platforms or not really.
Does it need to reinitialize for each request or is there a warm start / cold start model like lambda? I don't really understand how you can charge per request.
There is no cold start! We keep your service hot all the time.
This is nice, and I've wanted this kind of thing repeatedly over the last 5 years! I think you often want to run little bits of CPU-based code in addition to your deep learning graph. So I think a better deployment model might be basically Lambda but with CUDA access... or something like that. The factors that I think would make this service most valuable are low cost (think, lower than GPU's on AWS or similar, even…
Thank you! We don't have any cold start delay! In our custom environment, you can do exactly what you are describing (running both CPU and GPU code). We provide you with access to the GPU and the CUDA libraries installed. It's basically lambda (minus the cold start) with GPU access. We can scale a lot very quickly depending on how much you need.
Are you willing to talk a bit about how this all works? I assume you host the hardware yourself somewhere, which in the days of AWS et al must be pretty tough to pull off, especially with these specs. Where do you get the hardware from these days with the crypto craze?