it looks like https://news.ycombinator.com/item?id=37248844 has gotten the traction at 295 points
Code Llama, a state-of-the-art large language model for coding
111–120 of 525 posts
Re: Code Llama, a state-of-the-art large language model for coding
#112It's really sad how everyone here is fawning over tech that will destroy you own livelihoods. "AI won't take your job, those who use AI will" is purely short term, myopic thinking. These tools are not aimed to help workers, the end goal is to make it so you don't need to be an engineer to build software, just let the project manager or director describe the system they want and boom there it is. You can scream that t…
Improve productivity, cheapen goods and services. Nature of technological advancement
Re: Code Llama, a state-of-the-art large language model for coding
#113Does anyone have a good explanation for Meta's strategy with AI? The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?
I watched a good talk from Yann LeCun who is Chief AI Scientist at Meta, and he explained that the thinking is that open source AI models will be the long-term winner, so it's best for them to work in that arena. https://www.youtube.com/watch?v=vyqXLJsmsrk
That's not a business strategy.
Likely this is driven by ego.
Yann wants to cement his position as a leader in AI and while he clearly does not appreciate LLMS at all, he realizes that he needs to make waves in this area.
Mark needs a generative product and has invested tremendously in the infrastructure for AI in general (for recommendation). He needs researchers to use that infrastructure to create a generative product(s).
Yann sees this going on, realizes that he has a very powerful (research+recruiting) position and tells mark that he will only sign on if Meta gives away a good deal of research and Mark concedes, with the condition that he wants his generative product by end of 2023 or start of 2024.
Re: Code Llama, a state-of-the-art large language model for coding
#114[deleted]
Re: Code Llama, a state-of-the-art large language model for coding
#115How are people using these local code models? I would much prefer using these in-context in an editor, but most of them seem to be deployed just in an instruction context. There's a lot of value to not having to context switch, or have a conversation. I see the GitHub copilot extensions gets a new release one every few days, so is it just that the way they're integrated is more complicated so not worth the effort?
http://cursor.sh integrates GPT-4 into vscode in a sensible way. Just swapping this in place of GPT-4 would likely work perfectly. Has anyone cloned the OpenAI HTTP API yet?
Re: Code Llama, a state-of-the-art large language model for coding
#116Earlier quoted context omitted.
I’ve found the utility of the coding LLMs gets a lot higher when you’ve got code comments and descriptive variable and function names - the LLM makes better inferences and suggestions. We’ve seen similar on data - properly tagged data and descriptive field names helps the LLM to produce much more useful responses. I’m secretly hoping the spread of these tools will finally lead my fellow developers to comment their co…
Commenting the code in this manner sounds like a job for an LLM, maybe with human assistance in the short run.
This is my ultimate (short term) AI fear - letting it get into a feedback loop with itself, leading to perverse and incorrect results.
To state my position more clearly: I don’t think an AI could comment code from scratch very well - how would it know all the decisions made, business logic considerations, historical conventions, micro-industry standards, etc?
A good benchmark I was told once was “if a human expert couldn’t do it, an AI probably can’t either”. And commenting code I didn’t write would certainly test the bounds of my abilities
Re: Code Llama, a state-of-the-art large language model for coding
#117Interesting that there's a 34B model. That was missing from the original Llama 2 release. I wonder if it's still usable for general non-code chat tasks or if the code fine tuning destroyed that. It should be the best model that would still fit on 24GB gaming GPUs with quantization, because 70B doesn't fit.
I can't imagine it being better than Llama1 33B, after all this code finetuning.
But the license for llama 2 is a whole lot better.
Re: Code Llama, a state-of-the-art large language model for coding
#118Does anyone have a good explanation for Meta's strategy with AI? The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?
the only beneficiary of this are the hardware vendors.. nvidia and amd. and startups which get these foundation models for free. because language models are a complementary product, and the complement must be commoditized as a strategy. I see AMD as a bigger beneficiary, since, very soon, amd will equal nvidia for inference and fine-tuning, but amd has a long way to go to equal in foundation model training.
> and startups which get these foundation models for free.
It's licensed non-commercially, so I'm not sure what those startups stand to gain.
> since, very soon, amd will equal nvidia for inference and fine-tuning
Source? If you're referring to Olive, it is indeed impressive but also has caveats:
1. It is just as proprietary as CUDA or CoreML.
2. You need a copy of Windows and licensed DirectX to use those optimizations.
3. AMD only matches Nvidia's inferencing performance when comparing Olive to Pytorch. Olive-to-Olive comparisons will still reflect an Nvidia lead.
I don't think AMD has the capability to equal Nvidia in the short-term. It will take longtime software investments from across the industry to shake Nvidia's yoke.
Re: Code Llama, a state-of-the-art large language model for coding
#119Earlier quoted context omitted.
Did Meta add scalable rope to the official implementation?
We changed RoPE's theta from 10k to 1m and fine-tuned with 16k tokens long sequences.
Curious, what led you to adjusting the parameters this way? Also, have you guys experimented with ALiBi[1] which claims better extrapolative results than rotary positional encoding?
[1]: https://arxiv.org/abs/2108.12409 (charts on page two if you’re skimming)
Re: Code Llama, a state-of-the-art large language model for coding
#120Code llama Python is very interesting. Specifically tuned for Python. I wonder if we could make such specific LLMs (one that is proficient in all things Rust, another- all things Linux, all things genomics, all things physics modeling etc) and have them talk to each other to collaboratively solve problems. That would be a crazy future thing! Putting machines truly to work..
Mark my words: you’ve caught a glimpse of the near future :). Google “Society of Mind” if you’re not yet familiar