11–20 of 50 posts
The results are pretty good; I wish they'd just publish the models so we can run the inference locally (not too many people have access to 8xA100 to train themselves, though I appreciate including the training data and instructions too).
The results are pretty good; I wish they'd just publish the models so we can run the inference locally (not too many people have access to 8xA100 to train themselves, though I appreciate including the training data and instructions too).
> The code runs on a 8xA100 80GB, but can also run on 8xA10040GB or 4xA100 with lower batch size and gradient accumulation steps. To get the GPUs, I suggest using Lambda Labs, best pricing for the best hardware. I wonder how much it was total in $ for the fine-tuning. Also, does anyone have some sort of table/formula that relates MB/GB of training data to $ for fine-tuning?
> The code runs on a 8xA100 80GB, but can also run on 8xA10040GB or 4xA100 with lower batch size and gradient accumulation steps. To get the GPUs, I suggest using Lambda Labs, best pricing for the best hardware. I wonder how much it was total in $ for the fine-tuning. Also, does anyone have some sort of table/formula that relates MB/GB of training data to $ for fine-tuning?
> The code runs on a 8xA100 80GB, but can also run on 8xA10040GB or 4xA100 with lower batch size and gradient accumulation steps. To get the GPUs, I suggest using Lambda Labs, best pricing for the best hardware. I wonder how much it was total in $ for the fine-tuning. Also, does anyone have some sort of table/formula that relates MB/GB of training data to $ for fine-tuning?
Stanford only spent $500 to fine-tune LLAMA for humam instruction with 52k instructions generated by GPT-3. This probably costs less. The use of GPT to generate the instruction data instead of humans is the massive cost reduction. The actual training for fine-tuning on GPUs is relatively cheap.
Earlier quoted context omitted.
Why would they?
To avoid the embarrassment of having to serve themselves with a DMCA?
(But you're right, those who send DMCAs are likely to just a send a link to the original repository :)
https://techcrunch.com/2022/12/20/petals-is-creating-a-free-...
could be modified to create a system that's trained.
> The code runs on a 8xA100 80GB, but can also run on 8xA10040GB or 4xA100 with lower batch size and gradient accumulation steps. To get the GPUs, I suggest using Lambda Labs, best pricing for the best hardware. I wonder how much it was total in $ for the fine-tuning. Also, does anyone have some sort of table/formula that relates MB/GB of training data to $ for fine-tuning?
Probably in the hundreds of dollar for 7B model, and may be a thousand or two for the 13B at worst
The repo above can be replicated for similar costs. Easily less than $10 for up to 30B using LoRA (which requires only 24GB of VRAM for 30B/33B and smaller).