I wish these releases had short but meaningful descriptions for their models.
E.g. 10 layers x 2048 tokens x 1024 embedding model using full attention.
Falcon 40B LLM (which beats Llama) now Apache 2.0
141–143 of 143 posts
Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#142It's interesting, that intruction-tuned model has lower performance on ARC and HellaSwag.
Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#143Earlier quoted context omitted.
Some people use second hand P40 GPUs, which go for around 200-300$. Combine 3 of them with SLI and you've got 72GB of VRAM for less then $1000
I do use a P40 for my machine learning box, but I'm curious how you put three on the same system, given they need a CPU power plug and a pci-e port. Then, to cool them, you need to plug your own cooling system, requiring more specific power plugs to be available. What kind of chassis, motherboard, power unit you use to do that? It'll certainly will cost more than $1000 anyway, especially since you also need a decent…
http://nonint.com/ has some interesting posts about how he build a custom server to house 8 GPU's (3090's in this case). You're right that that will set you back more than $1000, though I was only referring to the GPU's themselves.