Hetzner introduces GPU server for AI training
1–10 of 57 posts
Re: Hetzner introduces GPU server for AI training
#2Although R2 and B3 are excellent alternatives to S3, go get them Hetzner!
Hetzner is a great, reliable company with fantastic offerings and excellent support.
Re: Hetzner introduces GPU server for AI training
#3Although R2 and B3 are excellent alternatives to S3, go get them Hetzner! Hetzner is a great, reliable company with fantastic offerings and excellent support.
I think you got your tabs mixed up, Hetzner did just launch an S3-like in beta but this post is about GPU boxes.
Re: Hetzner introduces GPU server for AI training
#4I always looks at these prices and think it’s a complete rip off for anyone doing less than 4 GPUs.
Re: Hetzner introduces GPU server for AI training
#5What’s the most cost effective option for hosting an llm these days? I don’t need to train, I just want to use one of the llama models for inference to reduce my reliance on 3rd parties.
Re: Hetzner introduces GPU server for AI training
#6Hmm. Seems like a bad deal.
This is a monthly reservation for a single 6000 Ada for $940. You can get the same on RunPod for $670.
And to actually train stuff you'd likely want nodes with more of them, like 8, or just different GPUs all together (like A100/H100/etc).
Re: Hetzner introduces GPU server for AI training
#7What’s the most cost effective option for hosting an llm these days? I don’t need to train, I just want to use one of the llama models for inference to reduce my reliance on 3rd parties.
Have you tried how far you will get with a Hetzner VPS with dedicated CPU(s)?
Re: Hetzner introduces GPU server for AI training
#8Hmm. Seems like a bad deal. This is a monthly reservation for a single 6000 Ada for $940. You can get the same on RunPod for $670. And to actually train stuff you'd likely want nodes with more of them, like 8, or just different GPUs all together (like A100/H100/etc).
How long I wonder before these sorts of things will be affordable to the more everyday person say sub-$100?
Re: Hetzner introduces GPU server for AI training
#9[dead]
Re: Hetzner introduces GPU server for AI training
#10What’s the most cost effective option for hosting an llm these days? I don’t need to train, I just want to use one of the llama models for inference to reduce my reliance on 3rd parties.
Depends on the specific model and your perf requirements, but lots of them will run on a single box with a middle of the road GPU. If your invocation rate is low, hosted solutions like AWS Bedrock or using hosted APIs might be cheaper.