Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RamaLama

github.com

Post reply on HN
Print view View original

41–50 of 54 posts

Re: RamaLama

#41

Post by creesch »

mchiang wrote: post #29
eigenvalue wrote: post #4

This is the point of it: https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...

I’m one of the maintainers of Ollama. It’s amazing to see others build on top of open-source projects. Forks like RamaLama are exactly what open source is all about. Developers with different design philosophies can still collaborate in the open for everyone’s benefit. Some folks on the Ollama team have contributed directly to the OCI spec, so naturally we started with tools we know best. But we made a conscious deci…

Since you are one of the maintainers of Ollama, maybe you can help me answer a related question. It is great that the software itself is open source, but hosting the models must cost a fortune. I know this is funded by VC money, yet nowhere on the Ollama website or repository there is any mention of this. Why is that?

There isn't an about section, a tiny snippet in a FAQ somewhere, nothing.

Re: RamaLama

#42

Post by sitkack »

guerrilla wrote: post #31
sitkack wrote: post #27

Earlier quoted context omitted.

So you have never hit the issue so no one else has?

... orrrrr I have never hit the issue, so that's why I'm asking. Calm down. It's Friday, time to relax, my friend. ;)

It seemed an awful lot like you were feigning confusion with a lack of empathy towards why someone would want to use a container to have repeatable environments.

https://mannerofspeaking.org/2013/03/23/rhetorical-devices-a...

I know you are just asking questions.

Re: RamaLama

#43

Post by justinmayer »

mchiang wrote: post #29
eigenvalue wrote: post #4

This is the point of it: https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...

I’m one of the maintainers of Ollama. It’s amazing to see others build on top of open-source projects. Forks like RamaLama are exactly what open source is all about. Developers with different design philosophies can still collaborate in the open for everyone’s benefit. Some folks on the Ollama team have contributed directly to the OCI spec, so naturally we started with tools we know best. But we made a conscious deci…

These comments would carry more merit if they weren’t coming from the very person who closed this pull request: https://github.com/jmorganca/ollama/pull/395

Those rejected README changes only served to provide greater transparency to would-be users, and here we are a year and a half later with woefully inadequate movement on that front.

I am very glad folks are working on alternatives.

Re: RamaLama

#44

Post by threecheese »

justinmayer wrote: post #43
mchiang wrote: post #29

Earlier quoted context omitted.

I’m one of the maintainers of Ollama. It’s amazing to see others build on top of open-source projects. Forks like RamaLama are exactly what open source is all about. Developers with different design philosophies can still collaborate in the open for everyone’s benefit. Some folks on the Ollama team have contributed directly to the OCI spec, so naturally we started with tools we know best. But we made a conscious deci…

These comments would carry more merit if they weren’t coming from the very person who closed this pull request: https://github.com/jmorganca/ollama/pull/395 Those rejected README changes only served to provide greater transparency to would-be users, and here we are a year and a half later with woefully inadequate movement on that front. I am very glad folks are working on alternatives.

As an outsider (not an oss maintainer, but a contributor), the decline to merge imo was understandable - the maintainer had a strategy and it didn’t fit. They gave reasons why - really nicely - and even made a call to action for PRs placing architecture docs elsewhere. Your response tonally was disparaging, and the subsequent pile on was anti productive. All due respect to your experience as a maintainer; in that role, can you imagine seeing a contribution that you are not interested in, and declining/forgetting to or being too busy to engage, imagining that it might get dropped or made better while you are busy with your priorities?

Putting myself in your shoes, I can see why you might be annoying at being ignored. Suggests this change is really important to you, and so my question would be why didn’t you follow the maintainers advice and add architecture docs?

Re: RamaLama

#45

Post by threecheese »

mohsen1 wrote: post #19
eigenvalue wrote: post #4

This is the point of it: https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...

I see! Now I understand why I need to create those useless `Modelfile` files... I'm glad there is a more open source alternative to Ollama now.

Interested to know why one is “more open source” than the other.

Re: RamaLama

#47

Post by rhatdan »

One of my primary goals of RamaLama was to allow users to move AI Models into containers, so they can be stored in OCI Registries. I believe there is going to be a proliferation of "private" models, and eventually "private" RAG data. (Working heavily in RAG support in RamaLama now.

Once you have private models and RAG, I believe you will want to run these models and data on edge devices in in Kubernetes clusters. Getting the AI Models and data into OCI content. Would allow us to take advantage of content signing, trust, mirroring. And make running the AI in production easier.

Also allowing users to block access to outside "untrusted" AI Models stored in the internet. Allow companies to only use "trusted" AI.

Since Companies already have OCI registries, it makes sense to store your AI Models and content in the same location.

Re: RamaLama

#48

Post by rhatdan »

rhatdan wrote: post #47

One of my primary goals of RamaLama was to allow users to move AI Models into containers, so they can be stored in OCI Registries. I believe there is going to be a proliferation of "private" models, and eventually "private" RAG data. (Working heavily in RAG support in RamaLama now. Once you have private models and RAG, I believe you will want to run these models and data on edge devices in in Kubernetes clusters. Get…

Bottom line we want to take advantage of the infrastructure created by Podman, Docker and Kubernetes.

Re: RamaLama

#50

Post by ecurtin »

ggerganov wrote: post #18
mckirk wrote: post #9

This looks great! While we're at it, is there already some kind of standardized local storage location/scheme for LLM models? If not, this project could potentially be a great place to set an example that others can follow, if they want. I've been playing with different runtimes (Ollama, vLLM) the last days, and I really would have appreciated better interoperability in terms of shared model storage, instead of every…

The llama.cpp tools and examples download the models by default to a OS-specific cache folder [0]. We try to follow the HF standard (as discussed in the linked thread), though the layout of the llama.cpp cache is not the same atm. Not sure about the plans for RamaLama, but it might be something worth to consider. [0] https://github.com/ggerganov/llama.cpp/issues/7252

In RamaLama an OCI container-like store is used (at least from the UX perspective it feels like that) for all models in RamaLama, it's protocol agnostic supports oci artefacts, huggingface, ollama, etc.
Post reply on HN