Llama-3.2-11B-Vision-Instruct does an excellent job extracting/answering questions from screenshots. It is even able to answer questions based on information buried inside a flowchart. How is this even possible??
magic
Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
151–160 of 343 posts
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#152Llama-3.2-11B-Vision-Instruct does an excellent job extracting/answering questions from screenshots. It is even able to answer questions based on information buried inside a flowchart. How is this even possible??
Oh, this is promising. It's not surprising to me: image models have been very oriented towards photography and scene understanding rather than understanding symbolic information in images (like text or diagrams), but I always thought that it should be possible to make the model better at the latter, for instance by training it more on historical handwritten documents.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#153Earlier quoted context omitted.
I question whether a 3B model can have “a lot of knowledge”.
From quizzing it a bit it has good knowledge but limited reasoning. For example it will tell you all about the life and death of Ho Chi Minh (and as far as I can verify factual and with more detail than what's in English Wikipedia), but when quizzed whether 2kg of feathers are heavier than 1kg of lead it will get it wrong. Though I wouldn't treat it as a domain expert on anything. For example when I asked about the s…
> it oversold Rust a bit and claimed Python had issues it doesn't actually have
So exactly like a human
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#154Earlier quoted context omitted.
llm is Simon's command line front-end to a lot of the llm apis, local and cloud-based. Along with aider-chat, it's my main interface to any LLM work -- it works well with a chat model, one-off queries, and piping text or output into a llm chain. For people who live on the command line, or are just put-off by web interfaces, it's a godsend. About the only thing I need to look further abroad for is when I'm working mul…
I use a fair amount of aider - what does Simon's solution offer that aider doesn't? I am usually using a mix of aider and the ChatGPT window. I use ChatGPT for one off queries that aren't super context heavy for my codebase, since pricing can still add up for the API and a lot of the times the questions that I ask don't really need deep context about what I'm doing in the terminal. But when I'm in flow state and I ne…
The llm command is a general-purpose tool for writing shell scripts that use an llm somehow. For example, generating some llm output and sending it though a Unix pipeline. You can also use it interactively if you like working on the command line.
It’s not specifically about chatting or helping you write code, though you could use it for that if you like.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#155llama3.2:3b-instruct-q8_0 is performing better than 3.1 8b-q4 on my macbookpro M1. It's faster and the results are better. It answered a few riddles and thought experiments better despite being 3b vs 8b. I just removed my install of 3.1-8b. my ollama list is currently: $ ollama list NAME ID SIZE MODIFIED llama3.2:3b-instruct-q8_0 e410b836fe61 3.4 GB 2 hours ago gemma2:9b-instruct-q4_1 5bfc4cf059e2 6.0 GB 3 days ago p…
For a second I read that as “ it just removed my install of 3.1-8b” :D
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#156I have to say that running this model locally I was pleasantly suprised how well it ran, it doesn't use as much resources and produce decent output, comparable to ChatGPT, it is not quite as OpenAI but for a lot of tasks, since it doesn't burden the computer, it can be used with local model.
Next I want to try to use Aider with it and see how this would work.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#157Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#158If anyone else is looking for the bigger models on ollama and wondering where they are, the Ollama blog post answered that for me. The are "coming soon" so they just aren't ready quite yet[1]. I was a little worried when I couldn't find them but sounds like we just need to be patient. [1]: https://ollama.com/blog/llama3.2
We're working on it. There are already draft PRs up in the GH repo. We're still working out some kinks though.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#159Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#160llama3.2:3b-instruct-q8_0 is performing better than 3.1 8b-q4 on my macbookpro M1. It's faster and the results are better. It answered a few riddles and thought experiments better despite being 3b vs 8b. I just removed my install of 3.1-8b. my ollama list is currently: $ ollama list NAME ID SIZE MODIFIED llama3.2:3b-instruct-q8_0 e410b836fe61 3.4 GB 2 hours ago gemma2:9b-instruct-q4_1 5bfc4cf059e2 6.0 GB 3 days ago p…
Aren't the _0 quantizations considered deprecated and _K_S or _K_M preferable?