11–20 of 31 posts
Amazon’s g6 instances are L4-based with 24gb vram, half the capacity of the L40S, with sagemaker in demand prices at this rate. Vast ai is cheaper, though a little more like bidding and varying in availability.
> You can run DOOM Eternal, building the Stadia that Google couldn’t pull off, because the L40S hasn’t forgotten that it’s a graphics GPU. Savage. I wonder if we’ll see a resurgence of cloud game streaming
I just had to implement GPU clustering in my inference stack to support Llama 3.1 70b, and even then I needed 2xA100 80GB SXMs.
I was initially running my inference servers on fly.io because they were so easy to get started with. But I eventually moved elsewhere because the prices were so high. I pointed out to someone there that e-mailed me that it was really expensive vs. others and they basically just waved me away.
For reference, you can get an A100 SXM 80GB spot instance on google cloud right now for $2.04/hr ($5.07 regular).
Not as fast as the L40S, but Runpod.io has the A40 48gb for $0.28/hr spot price, so if its mainly VRAM you need, this is a lot cheaper option. Vast.ai has it for the same price as well.
L40S has 48GB of RAM, curious how they're able to run Llama 3.1 70B on it. The weights alone would exceed this. Maybe they mean quantized/fp8? I just had to implement GPU clustering in my inference stack to support Llama 3.1 70b, and even then I needed 2xA100 80GB SXMs. I was initially running my inference servers on fly.io because they were so easy to get started with. But I eventually moved elsewhere because the pr…
Suddenly cutting prices in half shows that the business model is in dire straits.
This all happened because we were having internal meetings about trying to find A10s to rack, and Kurt stopped and said "wtf are we doing".
If it'll make you feel better, we'll continue to charge you the previous list price for L40S GPU hours.
Prices lowered to $1.25/hr... still 2X vast.ai prices.
There are definitely GPU providers where you can buy cheaper L40S hours than us. I'm not entirely sure what their system architectures are, or whether they're just buying in absolutely spectacular volume, because we are cutting pretty close to the bone with our pricing. One cost factor we have that other providers might not have (I'd love to know): we have to dedicate individual racked physical hosts to each group of…
Ya, that's a no from me.
> You can run DOOM Eternal, building the Stadia that Google couldn’t pull off, because the L40S hasn’t forgotten that it’s a graphics GPU. Savage. I wonder if we’ll see a resurgence of cloud game streaming
> You can run DOOM Eternal, building the Stadia that Google couldn’t pull off, because the L40S hasn’t forgotten that it’s a graphics GPU. Savage. I wonder if we’ll see a resurgence of cloud game streaming
Is the services that PlayStation Now uses publicly known? That's the only streaming service I've used so far.