Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RamaLama

github.com

Post reply on HN
Print view View original

31–40 of 54 posts

Re: RamaLama

#31

Post by guerrilla »

sitkack wrote: post #27
guerrilla wrote: post #8

> Running in containers eliminates the need for users to configure the host system for AI. When is that a problem? Based on the linked issue in eigenvalue's comment[1], this seems like a very good thing. It sounds like ollama is up to no good and this is a good drop-in replacement. What is the deeper problem being solved here though, about configuring the host? I've not run into any such issue. 1. https://news.ycombi…

So you have never hit the issue so no one else has?

... orrrrr I have never hit the issue, so that's why I'm asking.

Calm down. It's Friday, time to relax, my friend. ;)

Re: RamaLama

#32

Post by zozbot234 »

mohsen1 wrote: post #19
eigenvalue wrote: post #4

This is the point of it: https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...

I see! Now I understand why I need to create those useless `Modelfile` files... I'm glad there is a more open source alternative to Ollama now.

I don't get it. The 'Modelfile' files are used to save and restore chat history as well, set custom system prompts and lots of other stuff that would require custom coding with most other local AI frameworks. Llama.cpp certainly doesn't offer anything like that out of the box. Those sorts of complaints seem pointless to me.

Re: RamaLama

#33

Post by jerrygenser »

122 points 2 hours ago yet this is currently #38 and not on the front page.

Strange. At the same time I see numerous items that are on the front page posted 2 hours or older with fewer points.

I'm willing to take a reputation hit on this meta post. I wonder why this got demoted so quickly from front page despite people clearly voting on it. I wonder if it has anything to do with being backed by YC.

I sincerely hope it's just my miss understanding of hn algorithm though

Re: RamaLama

#34

Post by zozbot234 »

mchiang wrote: post #29
eigenvalue wrote: post #4

This is the point of it: https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...

I’m one of the maintainers of Ollama. It’s amazing to see others build on top of open-source projects. Forks like RamaLama are exactly what open source is all about. Developers with different design philosophies can still collaborate in the open for everyone’s benefit. Some folks on the Ollama team have contributed directly to the OCI spec, so naturally we started with tools we know best. But we made a conscious deci…

These comments seem reasonable to me. Could you clarify the Ollama maintainers' POV wrt. the recent discussion of Ollama Vulkan support at https://news.ycombinator.com/item?id=42886680 ? Many people seem to be upset that this PR seems to have gotten zero acknowledgment from the Ollama folks, even with so many users being quite interested in it for obvious reasons. (To be clear, I'm not sure that the PR is in a mergeable state as-is, so I would disagree with many of those comments. But this is just my personal POV - and with no statement on the matter from the Ollama maintainers, users will be confused.)

EDIT: I'm seeing a newly added comment in the Vulkan PR GitHub thread, at https://github.com/ollama/ollama/pull/5059#issuecomment-2628... . Quite overdue, but welcome nonetheless!

Re: RamaLama

#35

Post by mchiang »

jerrygenser wrote: post #33

122 points 2 hours ago yet this is currently #38 and not on the front page. Strange. At the same time I see numerous items that are on the front page posted 2 hours or older with fewer points. I'm willing to take a reputation hit on this meta post. I wonder why this got demoted so quickly from front page despite people clearly voting on it. I wonder if it has anything to do with being backed by YC. I sincerely hope i…

Can confirm it doesn't. Many Ollama posts get pushed off the front page too despite having hundreds of points. Over time I understood. If they did this for YC companies, it would ruin the trust of HN, YC, and probably the most important to YC companies, the reputation of the startup itself.

Re: RamaLama

#36

Post by zozbot234 »

mchiang wrote: post #35
jerrygenser wrote: post #33

122 points 2 hours ago yet this is currently #38 and not on the front page. Strange. At the same time I see numerous items that are on the front page posted 2 hours or older with fewer points. I'm willing to take a reputation hit on this meta post. I wonder why this got demoted so quickly from front page despite people clearly voting on it. I wonder if it has anything to do with being backed by YC. I sincerely hope i…

Can confirm it doesn't. Many Ollama posts get pushed off the front page too despite having hundreds of points. Over time I understood. If they did this for YC companies, it would ruin the trust of HN, YC, and probably the most important to YC companies, the reputation of the startup itself.

I assume this is what happens when many HN users just flag every AI- and LLM-related post out of sheer frustration with the reality distortion field around this particular topic.

Re: RamaLama

#37

Post by svilen_dobrev »

mckirk wrote: post #9

This looks great! While we're at it, is there already some kind of standardized local storage location/scheme for LLM models? If not, this project could potentially be a great place to set an example that others can follow, if they want. I've been playing with different runtimes (Ollama, vLLM) the last days, and I really would have appreciated better interoperability in terms of shared model storage, instead of every…

i just started to play with ollama and ramalama.. on linux. The models are quite some gigabytes.. not pretty to keep N copies..

ollama stores things under ~/.ollama/models/blobs/ named sha256-whatevershaisit

ramalama stores things under ~/.local/share/ramalama/repos/ollama/blobs/ named sha256:whatevershaisit

Note the ":" in ramalama names instead of the "-" .. that may not fly under windows.

if one crosslinks ramalama things over to ollama with that slight rename, ollama will remove them as they are not pulled via itself - no metadata on them.

i guess vllm etc everybody-else has yet-another schema and/or metadata.

btw Currently, arch-linux-wise, there is llm-manager (pointing to https://github.com/xyproto/llm-manager ), but it's made dependent on some of ollama packages, and can't be installed just by itself (without overforcing).

Re: RamaLama

#38

Post by baron-bourbon »

balloob wrote: post #25
baron-bourbon wrote: post #20

Does this provide a Ollama compatible API endpoint? I've got at least one other project running that only supports Ollama's API or OpenAI's hosted solution (ie. the API endpoint isn't configurable to use llama.cpp and friends)

We need to stop chasing compatible API endpoints and work towards an AI standard. I wrote about it here https://news.ycombinator.com/item?id=42887610

I agree with what you wrote. The whole situation reminds me of the old "Standards" XKCD, to an extent. In the short term something like LiteLLM, which I just discovered doing more research on the whole topic, can at least hide some of the underlying complexity.

That being said, considering what you've done with Open Home and Home Assistant (which has run my home for years, thank you!), perhaps there is some hope of an open standard in the near future.

Re: RamaLama

#39

Post by [deleted] »

mchiang wrote: post #29
eigenvalue wrote: post #4

This is the point of it: https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...

I’m one of the maintainers of Ollama. It’s amazing to see others build on top of open-source projects. Forks like RamaLama are exactly what open source is all about. Developers with different design philosophies can still collaborate in the open for everyone’s benefit. Some folks on the Ollama team have contributed directly to the OCI spec, so naturally we started with tools we know best. But we made a conscious deci…

[deleted]

Re: RamaLama

#40

Post by homarp »

zozbot234 wrote: post #32
mohsen1 wrote: post #19

Earlier quoted context omitted.

I see! Now I understand why I need to create those useless `Modelfile` files... I'm glad there is a more open source alternative to Ollama now.

I don't get it. The 'Modelfile' files are used to save and restore chat history as well, set custom system prompts and lots of other stuff that would require custom coding with most other local AI frameworks. Llama.cpp certainly doesn't offer anything like that out of the box. Those sorts of complaints seem pointless to me.

you tried a recent build of llama-server (from llama.cpp) ? the web interface does remember my chat, and obviously let me change all settings.
Post reply on HN