What's a good and friendly place to start for a seasoned Android developer to get a peek into the world of AI/ML as we are seeing all this today (or specifically last few months maybe), who is kind of having a FOMO cum curiosity (and a little worry about career/future and all) about all this? To get a taste of things.
Beginner's Guide to Llama Models
31–40 of 41 posts
Re: Beginner's Guide to Llama Models
#32An easy way to try many of the fine-tuned Llama 2 models is https://github.com/jmorganca/ollama . A maintainer of the project has been collecting a full list here (with different quantization levels), most of which are Llama 2-based: https://gist.github.com/mchiang0610/b959e3c189ec1e948e4f6a1f... Since the release of Llama 2 the number of models based on it has been growing significantly.. some popular ones: - codeup…
Wow, thanks! I really like Ollama, so I'm glad the models listed at the top of the Readme aren't all you could use. Do you know where the data in that Gist came from? Is there a registry somewhere?
There is! While not easy to use yet, there's a sort-of-hidden way models can be listed with:
curl https://ollama.ai/v2/_catalog | jq
curl https://ollama.ai/v2/library/llama2/tags/list | jqRe: Beginner's Guide to Llama Models
#33What are the hardware requirements to fine tune llama ?
~20 GB vram for the 7B model and 48 GB for the 13B model.
It depends on the context size as well. I'd recommend renting a 4090 from a cloud provider like runpod/vast ai to get started, using a PEFT tutorial.
Re: Beginner's Guide to Llama Models
#34What are the hardware requirements to fine tune llama ?
~20 GB vram for the 7B model and 48 GB for the 13B model. It depends on the context size as well. I'd recommend renting a 4090 from a cloud provider like runpod/vast ai to get started, using a PEFT tutorial.
Thanks. What about the 70B model? I assume a 4090 will not be enough. Is it linear system requirements ?
Re: Beginner's Guide to Llama Models
#35Earlier quoted context omitted.
~20 GB vram for the 7B model and 48 GB for the 13B model. It depends on the context size as well. I'd recommend renting a 4090 from a cloud provider like runpod/vast ai to get started, using a PEFT tutorial.
Thanks. What about the 70B model? I assume a 4090 will not be enough. Is it linear system requirements ?
4090 only has 24 GB and will only be able to fine tune (and merge, which is more memory intensive) the 7B model. The RTX6000 with 48 GB is able to fine tune the 13B model. The 70B model presumably needs multiple GPUs, like 4 RTX6000.
For people starting out, you can also use a free GPU from Google colab to fine tune a 7B model. Finetuning 70B gets more expensive and I would suggest trying smaller models first with a high quality dataset.
It is mostly linear I think.
Re: Beginner's Guide to Llama Models
#36I find the whole site being covered in images of skinny waifu girls... offputting. The guide itself seems fine, if high-level. [This]( https://chat.lmsys.org/?leaderboard ) is a really nice link I hadn't seen before.
I don't mind waifu girls, but I really don't know how these images are related to Llama. It's not an article about Stable Diffusion right...? Huge content farm vibe. Edit: I read the article carefully. Yeah, not just content farm vibe. It's a content farm. > What can you do with Llama models? > You can use Llama models the same ways you use ChatGPT. > Chat. Just ask questions about things you want to know. > Coding.…
Maybe they just like anime a lot? Hanlon's razor and all.
Re: Beginner's Guide to Llama Models
#37I find the whole site being covered in images of skinny waifu girls... offputting. The guide itself seems fine, if high-level. [This]( https://chat.lmsys.org/?leaderboard ) is a really nice link I hadn't seen before.
I don't mind waifu girls, but I really don't know how these images are related to Llama. It's not an article about Stable Diffusion right...? Huge content farm vibe. Edit: I read the article carefully. Yeah, not just content farm vibe. It's a content farm. > What can you do with Llama models? > You can use Llama models the same ways you use ChatGPT. > Chat. Just ask questions about things you want to know. > Coding.…
I only got a couple paragraphs in before thinking, "This was written by a fluff AI, and probably not even Llama." I think Llama has better English proficiency.
Re: Beginner's Guide to Llama Models
#38Earlier quoted context omitted.
Thanks. What about the 70B model? I assume a 4090 will not be enough. Is it linear system requirements ?
4090 only has 24 GB and will only be able to fine tune (and merge, which is more memory intensive) the 7B model. The RTX6000 with 48 GB is able to fine tune the 13B model. The 70B model presumably needs multiple GPUs, like 4 RTX6000. For people starting out, you can also use a free GPU from Google colab to fine tune a 7B model. Finetuning 70B gets more expensive and I would suggest trying smaller models first with a…
Thanks. My plan is to use this research cluster: https://www.ex3.simula.no/resources
I will probably train how to fine tune on the small model but I don’t really need to use a worse model to save money.
Re: Beginner's Guide to Llama Models
#39I tried multiple flavors of llama models, they are all quite dumb. Even the 70b parameter one. It knows about more things which the smaller models just hallucinate when asked, but still cannot do even slightly more complex tasks. I'm also not sure about the current testing methodologies i.e. the 'passed the SAT' hype. Given that the training set already contains much of the information, we should probably compare the…
Post us some sample prompts and answers please.
Re: Beginner's Guide to Llama Models
#40Earlier quoted context omitted.
Wow, thanks! I really like Ollama, so I'm glad the models listed at the top of the Readme aren't all you could use. Do you know where the data in that Gist came from? Is there a registry somewhere?
There is! While not easy to use yet, there's a sort-of-hidden way models can be listed with: curl https://ollama.ai/v2/_catalog | jq Then to list "tags" for a given model (e.g. llama2): curl https://ollama.ai/v2/library/llama2/tags/list | jq
Oh, you're the maintainer! Hahaha my bad. Thanks :)