Can anyone recommend a webUI client for ollama?
ive been using https://github.com/valiantlynx/ollama-docker which comes with https://github.com/open-webui/open-webui
Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
171–180 of 343 posts
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#172Llama3.2 3B feels a lot better than other models with same size (e.g. Gemma2, Phi3.5-mini models). For anyone looking for a simple way to test Llama3.2 3B locally with UI, Install nexa-sdk( https://github.com/NexaAI/nexa-sdk ) and type in terminal: nexa run llama3.2 --streamlit Disclaimer: I am from Nexa AI and nexa-sdk is an open-sourced. We'd love your feedback.
It's a great tool. Thanks!
I had to test it with Llama3.1 and was really easy. At a first glance Llama3.2 didn't seem available. The command you provided did not work, raising "An error occurred while pulling the model: not enough values to unpack (expected 2, got 1)".
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#173This is great! Does anyone know if the llama models are trained to do function calling like openAI models are? And/or are there any function calling training datasets?
Yes (rationale: 3.1 was, would be strange to rollback.) In general, you'll do a ton of damage by constraining token generation to valid JSON - I've seen models as small as 800M handle JSON with that. It's ~impossible to train constraining into it with remotely the same reliability -- you have to erase a ton of conversational training that makes it say ex. "Sure! Here's the JSON you requested:"
What kind of damage is done by constraining token generation to valid JSON?
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#174[deleted]
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#175Llama-3.2-11B-Vision-Instruct does an excellent job extracting/answering questions from screenshots. It is even able to answer questions based on information buried inside a flowchart. How is this even possible??
Because they trained the text model. Then froze the weights. Then trained a vision model on text image pairs of progressively higher quality. Then trained an adapter to align their latent spaces. So it became smart on text then gain a new input sense magically without changing its weights
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#176Can anyone recommend a webUI client for ollama?
Open webui has promising aspects, the same authors are pushing for "pipelines" which are a standard for how inputs and outputs are modified on the fly for different purposes.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#177Earlier quoted context omitted.
Can it speak foreign languages like German, Spanish, Ancient Greek?
Yes. It can converse perfectly normal in German. However when quizzed about German idioms it hallucinates them (in fluent German). Though that's the kind of stuff even larger models often have trouble with. For example if you ask GPT 4 about jokes in German it will give you jokes that depend on word play that only works when translated to English. In normal conversation Llama seems to speak fluent German For Ancient…
Thank you very much for taking your time.
Your findings are Amazing! I have used ChatGPT to proofread compositions in German and French lately, but it would have never occurred to me that I should have tested ability to understand idioms, which are the cherry on the cake. I’ll have it a go
As for Ancient Greek or Latin, ChatGPT has provided consistent translations and great explanations but its compositions had errors that prevented me from using it in the classroom.
All in all, chatGPT is a great multilingual and polyglot dictionary and I’d be glad if I could even use it offline for more autonomy
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#178Off topic/meta, but the Llama 3.2 news topic received many, many HN submissions and upvotes but never made it to the front page: the fact that it's on the front page now indicates that moderators intervened to rescue it: https://news.ycombinator.com/from?site=meta.com (showdead on) If there's an algorithmic penalty against the news for whatever reason, that may be a flaw in the HN ranking algorithm.
Yeah I submitted what turned out to be a dupe but I could never find the original, probably was buried at the time. Then a few hours later it miraculously (re?)appeared.
AIUI exact dupes just get counted as upvotes, which hasn’t happened in my case.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#179Surprised no mention of audio?
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#180Can it run with llama-cpp-python?
If so, where can we find and download the gguf files? Are they distributed directly by meta, or are they converted to gguf format by third parties?