Can anyone recommend a webUI client for ollama?
openwebui
Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
91–100 of 343 posts
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#92Earlier quoted context omitted.
The llama 3.0, 3.1, & 3.2 all use the TikToken tokenizer which is the open source openai tokenizer.
GP is talking about context windows, not the number of token used by the tokenizer.
Somewhat confusingly, it appears the tokenizer vocabulary as well as the context length are both 128k tokens!
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#93Earlier quoted context omitted.
Do they tell you what training data they use for alignment? As in, what biases they intentionally put in the system they’re widely deploying?
Do you have some concrete example of biases in their models? Or are you just fishing for something to complain about?
Even without intentionally biasing the model, without knowing the biases that exist in the training data, they're just biased black boxes that come with the overhead of figuring out how it's biased.
All data is biased, there's no avoiding that fact.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#94Newbie question, what size model would be needed to have a 10x software engineer skills and no knowledge of the human kind (ie, no need to know how to make a pizza or sequence your DNA). Is there such a model?
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#95Earlier quoted context omitted.
That doesn't address the thing they're skeptical about, which is how much knowledge can be encoded in 3B parameters. 3B models are great for text manipulation, but I've found them to be pretty bad at having a broad understanding of pragmatics or any given subject. The larger models encode a lot more than just language in those 70B+ parameters.
Ok, but what we are probably debating is knowledge versus wisdom. Like, if I know 1+1 = 2, and I know the numbers 1 through 10, my knowledge is just 11, but my wisdom is infinite in the scope of integer addition. I can find any number, given enough time. I'm pretty sure the AI guys are well aware of which types of models they want to produce. Models that can intake knowledge and intelligently manipulate it would mean…
I don't think this is right.
Usually the problem is much simpler with small models: they have less factual information, period.
So they'll do great at manipulating text, like extraction and summarization... but they'll get factual questions wrong.
And to add to the concern above, the more coherent the smaller models are, the more likely they very competently tell you wrong information. Without the usual telltale degraded output of a smaller model it might be harder to pick out the inaccuracies.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#96Newbie question, what size model would be needed to have a 10x software engineer skills and no knowledge of the human kind (ie, no need to know how to make a pizza or sequence your DNA). Is there such a model?
No, not yet. And such LLM wouldn't speak back in English or French without some "knowledge of the human kind" as you put it.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#97Can anyone recommend a webUI client for ollama?
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#98Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#99I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…
I saw that you mention https://github.com/simonw/llm/. Hadn't seen this before. What is its purpose? And why not use ollama instead?
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#100I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…
I saw that you mention https://github.com/simonw/llm/ . Hadn't seen this before. What is its purpose? And why not use ollama instead?
llm is Simon's command line front-end to a lot of the llm apis, local and cloud-based. Along with aider-chat, it's my main interface to any LLM work -- it works well with a chat model, one-off queries, and piping text or output into a llm chain. For people who live on the command line, or are just put-off by web interfaces, it's a godsend.
About the only thing I need to look further abroad for is when I'm working multi-modally -- I know Simon and the community are mainly noodling over the best command line UX for that: https://github.com/simonw/llm/issues/331