11–20 of 225 posts
Does anyone know of a link to a codegen comparison page? In other words, you write your request, and it's submitted to multiple codegen engines, so you can compare the output.
GPT3.5+ prioritized code very heavily - there's no CodeGPT, its just GPT4, and every version is better than the last.
Whereas the Llama/Mistral models are now shipping the general language model first, then adding CodeLlama/Codestral with additional pretraining (it seems like we don't know how much more tokens are on this one, but CodeLLama was 500B-1T extra tokens of code).
Zuck has mentioned recently that he doesnt see coding ability as important for his usecases, whereas obviously OpenAI is betting heavily on code as a way to improve LLM reasoning for AGI.
chat.mistral.ai
This is an open weights 22B model. The download on Huggingface is 44GB. Is there a rule-of-thumb estimate for how much RAM this would need to be used locally? Is the RAM requirement the same for a GPU and "unified" RAM like Apple silicon?
you need enough RAM and HBM (GPU RAM) so it’s a constraint on both.
How does this compare to Github Copilot? It's not shown in their comparison
Edit: For those who don't believe me, https://github.com/microsoft/vscode-copilot-release/issues/6.... Gpt-4 for chat, 3.5 for code.
The abundance of programming is going to allow almost everyone to become a great programmer.
This is so exciting to see and each day programming is becoming a solved problem so we can focus on other things.
i've been noticing that there's a divergence in philosophy between Llama style LLMs (Mistral are Meta alums so I'm counting them in tehre) and OpenAI/GPT style LLMs when it comes to code. GPT3.5+ prioritized code very heavily - there's no CodeGPT, its just GPT4, and every version is better than the last. Whereas the Llama/Mistral models are now shipping the general language model first, then adding CodeLlama/Codestra…
That's a really surprising thing to hear, where did you see that? The only quote I've seen is this one:
>“One hypothesis was that coding isn’t that important because it’s not like a lot of people are going to ask coding questions in WhatsApp,” he says. “It turns out that coding is actually really important structurally for having the LLMs be able to understand the rigor and hierarchical structure of knowledge, and just generally have more of an intuitive sense of logic.”
https://www.theverge.com/2024/1/18/24042354/mark-zuckerberg-...
This is an open weights 22B model. The download on Huggingface is 44GB. Is there a rule-of-thumb estimate for how much RAM this would need to be used locally? Is the RAM requirement the same for a GPU and "unified" RAM like Apple silicon?
When the model gets quantized to say 4bit ints, it'll be 22B params * 0.5 bytes = 11GB for example.
This is an open weights 22B model. The download on Huggingface is 44GB. Is there a rule-of-thumb estimate for how much RAM this would need to be used locally? Is the RAM requirement the same for a GPU and "unified" RAM like Apple silicon?
B: number of parameters
Q: quantization (16 = no quantization)