Hetzner introduces GPU server for AI training

Post by lugao » Wed, Sep 25, 2024, 12:45 PM UTC

rafaelmn wrote: post #21

jsheard wrote: post #17

Earlier quoted context omitted.

Even when it does, GPUs with more VRAM than a flagship gaming model have always and will always come at a massive price premium. That ceiling is currently 24GB, if you need more than that it's going to cost you.

I don't see why. AMD could buy a lot of free community work just by putting out a 32 GB version of their next gen flagship at around 1k. Not chasing the margin for a generation would buy them a decent amount of community support - they need it if they want to compete with Nvidia at any point. Ahh I just saw OP was saying sub 100$ - yeah that's never going to happen.

Post by dindresto » Wed, Sep 25, 2024, 12:48 PM UTC

reitzensteinm wrote: post #30

dindresto wrote: post #29

Earlier quoted context omitted.

That's incorrect, dedicated Hetzner servers do not have hourly pricing. That only applies to Hetzner Cloud.

That's incorrect. Dedicated servers do now offer hourly pricing. See this page which lists the hourly pricing for the new GEX130 server: https://www.hetzner.com/dedicated-rootserver/matrix-gpu/ But, as far as I know (hence the sibling question), you still need to pay the setup fee each time you launch.

Post by elaus » Wed, Sep 25, 2024, 12:50 PM UTC

kirab wrote: post #25

mromanuk wrote: post #22

I use runpod or vast for training my (small) models (a few million parameters) mostly using RTX4090 up to 4 GPUs. Training is a sporadic task. Is not worth it for me to book it monthly (at these prices)

Hetzner customer here. It’s a little hard to understand in the UX, but the price shown is the monthly max price. It is actually paid by the hour. The price per hour for this server is € 1.5980 more info: https://docs.hetzner.com/general/others/new-billing-model/

Post by kirab » Wed, Sep 25, 2024, 12:58 PM UTC

dindresto wrote: post #34

reitzensteinm wrote: post #30

Earlier quoted context omitted.

That's incorrect. Dedicated servers do now offer hourly pricing. See this page which lists the hourly pricing for the new GEX130 server: https://www.hetzner.com/dedicated-rootserver/matrix-gpu/ But, as far as I know (hence the sibling question), you still need to pay the setup fee each time you launch.

Actually rechecking the terms of the GPU server, it states: > Cancellation period: 30 days to the end of the month To me that indicates it's not possible to order the server for only a few hours.

Post by exe34 » Wed, Sep 25, 2024, 1:03 PM UTC

loughnane wrote: post #5

What’s the most cost effective option for hosting an llm these days? I don’t need to train, I just want to use one of the llama models for inference to reduce my reliance on 3rd parties.