According to the benchmarks: https://github.com/mag-/gpu_benchmark
RTX 6000 Ada is ~A100
Hetzner introduces GPU server for AI training
51–57 of 57 posts
Re: Hetzner introduces GPU server for AI training
#52What’s the most cost effective option for hosting an llm these days? I don’t need to train, I just want to use one of the llama models for inference to reduce my reliance on 3rd parties.
Consider also an online llama as a service like deepinfra. I have a local 3090 for playing around with the smaller models, but it's nice having the option of calling the 405b.
Ooh, I like that. Can see using them as a stepping stone where I'm using an open source model but without the hassle of setting up my own machine (but that I could later).
Re: Hetzner introduces GPU server for AI training
#53CoCalc offers On-Demand GPU servers with H100s starting at $2.01 per hour (metered per second) through its integration with Hyperstack... It also has more budget-friendly options, like RTX A4000s at $0.18 per hour.
https://cocalc.com/features/compute-server
In case you are not familiar, CoCalc is a real-time collaborative environment for education and research that you can access via your web browser at https://cocalc.com/
Re: Hetzner introduces GPU server for AI training
#54Earlier quoted context omitted.
Actually rechecking the terms of the GPU server, it states: > Cancellation period: 30 days to the end of the month To me that indicates it's not possible to order the server for only a few hours.
That term appears on all servers, but seems to be in conflict with their billing FAQ: https://docs.hetzner.com/general/others/new-billing-model/ Either way... it's a mess. They added hourly billing for dedicated servers six months ago, there's not much excuse for still having contradictory information hanging around.
Yes, the GEX130 has a *one-time* setup fee. And yes, it is possible to cancel it on an hourly basis in most situations. For example, if you only had it for 3 days of a billing period, you would pay the hourly rate. We are currently in the process of updating our billing systems. There are a few situations where the 30 day to the end of the month policy still applies. For most situations the current cancellation process is more generous and flexible for the customer. And for the rest, it has stayed the same. If customers want to stop paying for a server, they need to cancel it.
We have a number of products that do not include setup fees, including our Server Auction dedicated servers and our cloud servers.
Re: Hetzner introduces GPU server for AI training
#55Earlier quoted context omitted.
That term appears on all servers, but seems to be in conflict with their billing FAQ: https://docs.hetzner.com/general/others/new-billing-model/ Either way... it's a mess. They added hourly billing for dedicated servers six months ago, there's not much excuse for still having contradictory information hanging around.
Yes, the GEX130 has a *one-time* setup fee. And yes, it is possible to cancel it on an hourly basis in most situations . For example, if you only had it for 3 days of a billing period, you would pay the hourly rate. We are currently in the process of updating our billing systems. There are a few situations where the 30 day to the end of the month policy still applies. For most situations the current cancellation proc…
To be frank, after reading your reply I'm still not 100% sure what the policy is. What are the situations where there are 30 days of notice? How does that interact with hourly billing? If I'm a week through the second month and cancel, does that mean I get charged 2 months 1 week plus the setup fee in total?
I'm a big fan of Hetzner and have used it off and on for ten years now, including auction, dedicated and cloud. I've had very few complaints over the years. But I stand by my post above - the information on your site is not clear and contradictory, and really needs to be cleaned up.
Re: Hetzner introduces GPU server for AI training
#56Earlier quoted context omitted.
Yes, the GEX130 has a *one-time* setup fee. And yes, it is possible to cancel it on an hourly basis in most situations . For example, if you only had it for 3 days of a billing period, you would pay the hourly rate. We are currently in the process of updating our billing systems. There are a few situations where the 30 day to the end of the month policy still applies. For most situations the current cancellation proc…
To be frank, after reading your reply I'm still not 100% sure what the policy is. What are the situations where there are 30 days of notice? How does that interact with hourly billing? If I'm a week through the second month and cancel, does that mean I get charged 2 months 1 week plus the setup fee in total? I'm a big fan of Hetzner and have used it off and on for ten years now, including auction, dedicated and cloud…
We are currently in the process of updating our billing system. A big part of that process is already done, but there are still some changes to come. That is why we have not yet updated our terms and conditions. The new policies are more generous to the customers, and for the most part allow to cancel most products on an hourly basis, including dedicated servers like the GEX130. --Katie
Re: Hetzner introduces GPU server for AI training
#57Earlier quoted context omitted.
To be frank, after reading your reply I'm still not 100% sure what the policy is. What are the situations where there are 30 days of notice? How does that interact with hourly billing? If I'm a week through the second month and cancel, does that mean I get charged 2 months 1 week plus the setup fee in total? I'm a big fan of Hetzner and have used it off and on for ten years now, including auction, dedicated and cloud…
We are currently in the process of updating our billing system. A big part of that process is already done, but there are still some changes to come. That is why we have not yet updated our terms and conditions. The new policies are more generous to the customers, and for the most part allow to cancel most products on an hourly basis, including dedicated servers like the GEX130. --Katie
That's great news. Thanks!