I wonder whether org-ai-mode could easily support this.
Code Llama, a state-of-the-art large language model for coding
71–80 of 525 posts
Re: Code Llama, a state-of-the-art large language model for coding
#72How are people using these local code models? I would much prefer using these in-context in an editor, but most of them seem to be deployed just in an instruction context. There's a lot of value to not having to context switch, or have a conversation.
I see the GitHub copilot extensions gets a new release one every few days, so is it just that the way they're integrated is more complicated so not worth the effort?
Re: Code Llama, a state-of-the-art large language model for coding
#73Earlier quoted context omitted.
what is that?
Facebook Connect is what used to be called Oculus Connect. Kinda their equivalent of Apple's WWDC, I guess. It's when and where the Quest 3 will be officially unveiled in full, for example.
Yep - here is the site: https://www.metaconnect.com/en/home
Re: Code Llama, a state-of-the-art large language model for coding
#74>The Code Llama models provide stable generations with up to 100,000 tokens of context. Not a bad context window, but makes me wonder how embedded code models would pick that context when dealing with a codebase larger than 100K tokens. And this makes me further wonder if, when coding with such a tool (or at least a knowledge that they’re becoming more widely used and leaned on), are there some new considerations tha…
This sounds like a job for middleware. Condensing split code into a single huge file, shortening comments, removing whitespace and such can be done by a preprocessor for the llm.
So now we need an llmpack like we did webpack? Could it be smart enough to truncate comments, white space, etc?
Re: Code Llama, a state-of-the-art large language model for coding
#75Does anyone have a good explanation for Meta's strategy with AI? The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?
Retention project to keep their top ML/AI staff engaged and not straying away? Working towards NLU that can solve content moderation once and for all? Contrast with tiktok which is clearly using word filters that are easily worked around with phrases like "un-alived" or "corn". They want to replace influencers and your friends with chatbots and keep you scrolling through an infinite feed of ads and AI generated conte…
A lot of top ML/AI talent has already bailed too, so some of it is probably them trying to keep open research closer to SOTA.
Re: Code Llama, a state-of-the-art large language model for coding
#76>The Code Llama models provide stable generations with up to 100,000 tokens of context. Not a bad context window, but makes me wonder how embedded code models would pick that context when dealing with a codebase larger than 100K tokens. And this makes me further wonder if, when coding with such a tool (or at least a knowledge that they’re becoming more widely used and leaned on), are there some new considerations tha…
That seems daft.
You can, I suppose, contract your code so that it’s context free and uses less tokens, but that makes it more confusing for humans and language models.
Taken to the extreme, you can see obviously with one letter functions and variables like i, j, k the model will be able to infer literally nothing and, thus, produce arbitrary nonsense.
Clearly the solution is to do what we already do to manage complexity which is to decompose large tasks into smaller black box modules with an api where the (large number of tokens) implementation is hidden and not known or relevant to using it.
If you give an LLM a function signature and good description, maybe some usage examples, it doesn’t need the implementation to use it.
Terseness decreases the ability of LLMs to process code; it doesn’t solve context length, and even at best it doesn’t scale.
100k tokens is plenty.
You don’t need to do anything like that.
Re: Code Llama, a state-of-the-art large language model for coding
#77Earlier quoted context omitted.
If you watch the Connect talks, I'll be speaking about this..
I wish that Meta would release models like SeamlessM4T[0] under the same license as llama2, or an even better one. I don't understand the rationale for keeping it under a completely non-commercial license, but I agree that is better than not releasing anything at all. There seem to be opportunities for people to use technology like SeamlessM4T to improve lives, if it were licensed correctly, and I don't see how any c…
Yeah - different projects have different goals and licenses aren't one size fits all. Depending on the project, type of technology, goals, etc.. we will select or even develop the right license that aligns with those goals. Hope this helps :)
Re: Code Llama, a state-of-the-art large language model for coding
#78no more work soon?
Re: Code Llama, a state-of-the-art large language model for coding
#79Looks like that we need to request the access first
In the past, LLAMA access was granted nearly immediately. For HuggingFace downloads, it took a full day.
Re: Code Llama, a state-of-the-art large language model for coding
#80>The Code Llama models provide stable generations with up to 100,000 tokens of context. Not a bad context window, but makes me wonder how embedded code models would pick that context when dealing with a codebase larger than 100K tokens. And this makes me further wonder if, when coding with such a tool (or at least a knowledge that they’re becoming more widely used and leaned on), are there some new considerations tha…
Your developer tool already maps out the entire code base in useful ways, such as knowing all the symbols available in the current context and the structure of classes. This information can be distilled for presentation to the LLM. For instance, if you’re wanting to generate a method implementation inside a C++ class, the LLM can be given a condensed version of the header files that the compiler would have access to on compiling that specific class. Removing white space and comments and boiling macros down saves a lot of tokens.
You can also probably skip including standard library headers since those will be well known to the LLM through its fine tuning.
Either way, consider that a typical preprocessed C++ file would push against the 100K limit even with some optimizations. You will definitely want to have some middleware doing additional refinement before presenting that file to the LLM.