Show HN: Ollama – Run LLMs on your Mac

What does this add over llama.cpp? Is it just an "easier" way to setup llama.cpp locally?

If so, I don't really get it, because setting up llama.cpp locally is quite easy and well documented. And this appears to be a fork. Seems a bit fishy to me, when looking at the other "top" comments (with this one having no upvotes, but still #2 right now).

(llama.cpp's original intention is identical to yours: The main goal of llama.cpp is to run the LLaMA model using 4-bit integer quantization on a MacBook¹)

¹ https://github.com/ggerganov/llama.cpp#description

The llama.cpp project is absolutely amazing. Our goal was to build with/extend the project (vs try to be an alternative). Ollama was originally inspired by the "server" example: https://github.com/ggerganov/llama.cpp/tree/master/examples/...

This project builds on llama.cpp in a few ways:

1. Easy install! Precompiled for Mac (Windows and Linux coming soon)

2. Run 2+ models: loading and unloading models as users need them, including via a REST API. Lots to do here, but even small models are memory hogs and they take quite a while to load, so the hope is to provide basic "scheduling"

3. Packaging: content-addressable packaging that bundles GGML-based weights with prompts, parameters, licenses and other metadata. Later the goal is to bundle embeddings and other larger files custom models (for specific use cases, a la PrivateGPT) would need to run.

edit: formatting

So far for me it's been nice not having to spend one second mucking with Python (which is slow, and can be complex managing venvs etc), but the Modelfile is what's started to pique my interest. Bundling up metadata about prompts, parameters, models and hopefully later embeddings, LoRas, etc seems really promising. (Considering most people are just sharing prompts around atm)

I added a PR that supports dynamic command output injection, among other interesting things: https://github.com/jmorganca/ollama/pull/132. So you can imagine taking the output of, say, top and having the LLM parse it into useful information for you. Or having a chat session where you roll a d20 between each user interaction to determine their luck. Then being able to share that around to other people generically.

I'm also hoping at some point this project or a similar one will enable easy passing of full blown pipelines/chains of stuff around, as well as a ChatGPT style conversation sync and sharing tool.

I like to troll LLM :-)

>>> if you drop a feather and a cannon ball together, at the same height, which one would hit the ground first?

Thank you for your question! I'm happy to help. However, I must point out that the question is based on a false premise. A feather and a cannon ball cannot be dropped together at the same height, as they have different weights and densities, which would cause them to fall at different rates. The feather would fall much slower than the cannon ball due to its light weight and lower density. So, to answer your question directly, the cannon ball would hit the ground first. But please keep in mind that this scenario is purely hypothetical, as it is not possible to drop a feather and a cannon ball together at the same height in real life. Is there anything else I can help you with?

>>> if you drop a feather and a cannon ball together, at the same height on the moon, which one would hit the ground first?

Thank you for your question! I'm happy to help. However, I must point out that the scenario you've described is not possible as the moon does not have an atmosphere, and therefore, there is no "ground" on the moon for either the feather or the cannon ball to hit. Both the feather and the cannon ball would remain suspended in mid-air due to the lack of gravity on the moon. So, neither one would hit the ground first. Is there anything else I can help you with?

Cartoon physics are inherently superior to reality