HuggingFace Training Cluster as a Service
huggingface.co
HuggingFace Training Cluster as a Service
1–10 of 47 posts
Re: HuggingFace Training Cluster as a Service
#2> Train your LLM at scale on our infrastructure
Is it really their infrastructure or are they using a cloud provider and this wraps it up and provides convenience for a price?
Re: HuggingFace Training Cluster as a Service
#3The other day when they announced more funding there was some speculation here about how they would make money, with someone suggesting it's by driving users to cloud gpu platforms (aws, azure). This support that, and it suggest where they will end up, i.e. as a front end for azure.
Re: HuggingFace Training Cluster as a Service
#4At the moment of writing the cost estimate for 70B multimodal model with 7T tokens on 1000 H100 GPUs is $18,461,354 with 184 days of training time.
Anyone willing to share an estimate how cost will come down each year as hardware keeps improving and possible new methodologies are found?
Personally I would not be surprised if it is possible to train the same dataset for half the cost 12 months from now.
Re: HuggingFace Training Cluster as a Service
#5At the moment of writing the cost estimate for 70B multimodal model with 7T tokens on 1000 H100 GPUs is $18,461,354 with 184 days of training time. Anyone willing to share an estimate how cost will come down each year as hardware keeps improving and possible new methodologies are found? Personally I would not be surprised if it is possible to train the same dataset for half the cost 12 months from now.
Well if you select "trainium nodes", it's already "only" $11,085,287.
Re: HuggingFace Training Cluster as a Service
#6I wonder what's the multimodal model, Flamingo?
Re: HuggingFace Training Cluster as a Service
#7> Train your LLM at scale on our infrastructure Is it really their infrastructure or are they using a cloud provider and this wraps it up and provides convenience for a price?
What’s the difference?
Re: HuggingFace Training Cluster as a Service
#8Re: HuggingFace Training Cluster as a Service
#9lowest price from the dropdowns...$43k
Re: HuggingFace Training Cluster as a Service
#10> Train your LLM at scale on our infrastructure Is it really their infrastructure or are they using a cloud provider and this wraps it up and provides convenience for a price?
I'd think "infrastructur" includes the nice front end and Python API that they have proven to be capable to pull off already.