[flagged]
Code Llama, a state-of-the-art large language model for coding
41–50 of 525 posts
Re: Code Llama, a state-of-the-art large language model for coding
#42Re: Code Llama, a state-of-the-art large language model for coding
#43Does anyone have a good explanation for Meta's strategy with AI? The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?
If you watch the Connect talks, I'll be speaking about this..
Excited! I hope your talks are just as informative as this comment. Keep rocking!
Re: Code Llama, a state-of-the-art large language model for coding
#44Earlier quoted context omitted.
>Even the 7B model of code llama seems to be competitive with Codex, the model behind copilot It's extremely good. I keep a terminal tab open with 7b running for all of my "how do I do this random thing" questions while coding. It's pretty much replaced Google/SO for me.
You've already downloaded and thoroughly tested the 7B parameter model of "code llama"? I'm skeptical.
Just sign up at meta and you'll get an email link in like 5 minutes
Re: Code Llama, a state-of-the-art large language model for coding
#45The highlight IMO > The Code Llama models provide stable generations with up to 100,000 tokens of context. All models are trained on sequences of 16,000 tokens and show improvements on inputs with up to 100,000 tokens. Edit: Reading the paper, key retrieval accuracy really deteriorates after 16k tokens, so it remains to be seen how useful the 100k context is.
Did Meta add scalable rope to the official implementation?
Re: Code Llama, a state-of-the-art large language model for coding
#46Earlier quoted context omitted.
You've already downloaded and thoroughly tested the 7B parameter model of "code llama"? I'm skeptical.
Just sign up at meta and you'll get an email link in like 5 minutes
Yes, that's not a response to my comment.
No one who has been using any model for just the past 30 minutes would say that it has "pretty much replaced Google/SO" for them, unless they were being facetious.
Re: Code Llama, a state-of-the-art large language model for coding
#47Feels like we're like a year away from local LLMs that can debug code reliably (via being hooked into console error output as well) which will be quite the exciting day.
Re: Code Llama, a state-of-the-art large language model for coding
#48So it’s stubborn, stinks, bites and spits?
No thanks, going back to Winamp.
Re: Code Llama, a state-of-the-art large language model for coding
#49Does anyone have a good explanation for Meta's strategy with AI? The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?
the only beneficiary of this are the hardware vendors.. nvidia and amd. and startups which get these foundation models for free.
because language models are a complementary product, and the complement must be commoditized as a strategy.
I see AMD as a bigger beneficiary, since, very soon, amd will equal nvidia for inference and fine-tuning, but amd has a long way to go to equal in foundation model training.