Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

github.com

Post reply on HN
Print view View original

11–20 of 65 posts

Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

#11

Post by wkat4242 »

brrrrrm wrote: post #9
thedatamonger wrote: post #2

this looks very awesome. can someone tell me why there is no chatter about this? is there something else out there that blows this out of the water in terms of ease of use and access to sample many LLM's ?

HN isnt really the best space for LLM news - r/LocalLlama and twitter are much better. I think HN has some cultural issues with “AI” news

Hmm I don't think so. Most comments are pretty positive.

I think the articles are just not really upvoted unless it's really big news, makes sense because HN is for more than just AI.

But I don't think it's anti-AI like most people here would be pretty anti-cryptocurrency (and for good reason IMO)

Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

#12

Post by CharlesW »

thedatamonger wrote: post #2

this looks very awesome. can someone tell me why there is no chatter about this? is there something else out there that blows this out of the water in terms of ease of use and access to sample many LLM's ?

I can recommend LM Studio and Msty if you're looking for something with an integrated UX.

Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

#13

Post by anotherpaulg »

I actually just benchmarked Llama3 70B coding with aider, and it did quite well. It scored similar to GPT 3.5.

You can use Llama3 70B with aider via Ollama [0]. It's also available for free via Groq [1] (with rate limits). And OpenRouter has it available [2] for low cost on their paid api.

[0] https://aider.chat/docs/llms.html#ollama

[1] https://aider.chat/docs/llms.html#groq

[2] https://aider.chat/docs/llms.html#openrouter

Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

#14

Post by sdesol »

jerrygenser wrote: post #10

I wonder if Ollama will or plans to have other "Supported backends" than llama.cpp. It's listed on the very last line of their readme as if the llama.cpp dependency is just incidental and a very minor detail rather than Ollama as a deployment mechanism for llama.cpp and gguf based models.

I don't think they will move away from llama.cpp until they are forced to. The number of people contributing to llama.cpp is quite significant [1] and it wouldn't make sense to use another backend given how quickly llama.cpp is iterating and growing.

[1] https://devboard.gitsense.com/ggerganov?r=ggerganov%2Fllama....

Full disclosure: This is my tool

Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

#15

Post by p1esk »

wkat4242 wrote: post #11
brrrrrm wrote: post #9

Earlier quoted context omitted.

HN isnt really the best space for LLM news - r/LocalLlama and twitter are much better. I think HN has some cultural issues with “AI” news

Hmm I don't think so. Most comments are pretty positive. I think the articles are just not really upvoted unless it's really big news, makes sense because HN is for more than just AI. But I don't think it's anti-AI like most people here would be pretty anti-cryptocurrency (and for good reason IMO)

I didn’t upvote it because I don’t use Ollama. To experiment with LLMs I use Huggingface. Does Ollama provide something I cannot get with Huggingface?

Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

#16

Post by gertop »

thedatamonger wrote: post #2

this looks very awesome. can someone tell me why there is no chatter about this? is there something else out there that blows this out of the water in terms of ease of use and access to sample many LLM's ?

LM Studio is a lot more user friendly, probably the easiest UI to use out there. No terminal nonsense, no manual to read. Just double click and chat. It even explains to you what the model names mean (eg diff between Q4_1 Q4_K Q4_K_M... For whatever reason all the other tools assume you know what it means).

Built-in model recommendations are also handy.

Very friendly tool!

However it's not open-source.

Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

#17

Post by gertop »

p1esk wrote: post #15
wkat4242 wrote: post #11

Earlier quoted context omitted.

Hmm I don't think so. Most comments are pretty positive. I think the articles are just not really upvoted unless it's really big news, makes sense because HN is for more than just AI. But I don't think it's anti-AI like most people here would be pretty anti-cryptocurrency (and for good reason IMO)

I didn’t upvote it because I don’t use Ollama. To experiment with LLMs I use Huggingface. Does Ollama provide something I cannot get with Huggingface?

Hugging face is a model repository.

Ollama allows you to run those models.

Different things.

Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

#18

Post by p1esk »

gertop wrote: post #17
p1esk wrote: post #15

Earlier quoted context omitted.

I didn’t upvote it because I don’t use Ollama. To experiment with LLMs I use Huggingface. Does Ollama provide something I cannot get with Huggingface?

Hugging face is a model repository. Ollama allows you to run those models. Different things.

I run models using HF just fine. I mean I’m using HF transformers repo, which gets models from HF hub.

Or do you mean commercial deployment of models for inference?

Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

#19

Post by simonw »

p1esk wrote: post #18
gertop wrote: post #17

Earlier quoted context omitted.

Hugging face is a model repository. Ollama allows you to run those models. Different things.

I run models using HF just fine. I mean I’m using HF transformers repo, which gets models from HF hub. Or do you mean commercial deployment of models for inference?

Are you talking about the Hugging Face Python libraries, the Hugging Face hosted inference APIs, the Hugging Face web interfaces, the Hugging Face iPhone app, Hugging Face Spaces (hosted Docker environments with GPU access) or something else?

Re: Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B

#20

Post by p1esk »

simonw wrote: post #19
p1esk wrote: post #18

Earlier quoted context omitted.

I run models using HF just fine. I mean I’m using HF transformers repo, which gets models from HF hub. Or do you mean commercial deployment of models for inference?

Are you talking about the Hugging Face Python libraries, the Hugging Face hosted inference APIs, the Hugging Face web interfaces, the Hugging Face iPhone app, Hugging Face Spaces (hosted Docker environments with GPU access) or something else?

I updated my comment above: I’m using HF transformers repo, which gets models from HF hub.
Post reply on HN