For people who directly want to check the benchmark - https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...
Falcon 40B LLM (which beats Llama) now Apache 2.0
61–70 of 143 posts
Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#62Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#63For people who directly want to check the benchmark - https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...
Very cool but how does this compare to GPT-4 (before it was nerfed)?
I feel like the best benchmark atm is the orig gpt-4 version.
Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#64Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#6540B is pretty large, right? I expect it would take 70GB or so of RAM to run it. That's some expensive hardware ($10,000 or more).
Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#66For people who directly want to check the benchmark - https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...
How does it compare to OpenAI's GPT4? It's not listed in the table.
Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#67Earlier quoted context omitted.
Very cool but how does this compare to GPT-4 (before it was nerfed)? I feel like the best benchmark atm is the orig gpt-4 version.
nerfed how? I didn't know this.
Prob. referring to this https://news.ycombinator.com/item?id=36134249
Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#68For people who directly want to check the benchmark - https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...
Very cool but how does this compare to GPT-4 (before it was nerfed)? I feel like the best benchmark atm is the orig gpt-4 version.
None of these models are even close to GPT 3.5, let alone GPT4.
Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#6940B is pretty large, right? I expect it would take 70GB or so of RAM to run it. That's some expensive hardware ($10,000 or more).
70GB of RAM would cost around $150 these days depending on how you get there. 64GB (2x32GB) of DDR4 is around $140 then another 8GB stick would be around $15.
Used DDR3 ECC would be roughly half that.
Re: Falcon 40B LLM (which beats Llama) now Apache 2.0
#7040B is pretty large, right? I expect it would take 70GB or so of RAM to run it. That's some expensive hardware ($10,000 or more).
With 4-bit quantization should take less than 40GB of VRAM/RAM.