51–60 of 107 posts
Running with a Local LLM on a Mac M1, this completely locked up my system for minutes. I tried to let it run, because the progress bar was ticking every now and then, but after 10 minutes I gave up and killed it.
Running with a Local LLM on a Mac M1, this completely locked up my system for minutes. I tried to let it run, because the progress bar was ticking every now and then, but after 10 minutes I gave up and killed it.
I wouldn't recommend unless you got at least 16gb ram (though possibly more is needed depending on what model is used).
Excited to see something already far more realized, and I’m looking forward to trying this out.
I’ve been working on a larger than small writing project using Obsidian, and my ultimate goal is to have conversations with the corpus of what I’ve written, and to use this to hone ideas and experiment with new ways of exploring the content.
Not sure if local LLMs are powerful enough yet to enable meaningful/reliable outcomes, but this is the kind of stuff that really excites me about the future of this tech.
Literally yesterday I spun up a project with the intent to build something exactly like this for Obsidian. Excited to see something already far more realized, and I’m looking forward to trying this out. I’ve been working on a larger than small writing project using Obsidian, and my ultimate goal is to have conversations with the corpus of what I’ve written, and to use this to hone ideas and experiment with new ways o…
https://github.com/zatevakhin/obsidian-local-llm
https://github.com/hinterdupfinger/obsidian-ollama
Which already exist and if nothing else are decent starting points.
> Not sure if local LLMs are powerful enough yet to enable meaningful/reliable outcomes
I've dabbled, briefly, with Ollama running Mistral locally on an M1 MacBook Pro with 32GB of unified memory, and throwing a couple of hundred markdown documents at it via a RAG resulted in quite decent output to prompts asking questions about abstract contents/summariesbbased on those docs.
So I'd say we're already at a point where you can have meaningful outcomes; reliability is a whole other issue though.
Great job! I played around with this on a couple of small knowledge bases using an open Hermes model I had downloaded. The “related notes” feature didn't provide much value in my experience, often the link was so weak it was nonsensical. The Q&A mode was surprisingly helpful for querying notes and providing overviews, but asking anything specific typically just resulted in less than helpful or false answers. I'm sure…
Thank you for your feedback! Working hard on improving the chunking to improve related notes section. RAG is fairly naive right now, with lots of improvements coming in the next few weeks.
IME, the prompt should be front/center in terms of importance and the key to unlocking the models potential. It's one of the main reasons why Textgen-Webui is sooooo good. You can really dial-in the prompt, from the template itself to working with the system message. Then begin futzing with the myriad of other parameters to achieve fantastic results.
Literally yesterday I spun up a project with the intent to build something exactly like this for Obsidian. Excited to see something already far more realized, and I’m looking forward to trying this out. I’ve been working on a larger than small writing project using Obsidian, and my ultimate goal is to have conversations with the corpus of what I’ve written, and to use this to hone ideas and experiment with new ways o…
There are these plugins: https://github.com/zatevakhin/obsidian-local-llm https://github.com/hinterdupfinger/obsidian-ollama Which already exist and if nothing else are decent starting points. > Not sure if local LLMs are powerful enough yet to enable meaningful/reliable outcomes I've dabbled, briefly, with Ollama running Mistral locally on an M1 MacBook Pro with 32GB of unified memory, and throwing a couple of hundr…
I recently got my hands on an RTX 3090 for my Linux workstation and I’m planning to try getting some kind of remote setup going for my MacBook Air.
Great to hear about decent output. Reliability is negotiable as long as there’s some value and hopefully a path to future improvements.
I downloaded:
mistral-7b-v0.1.Q4_K_M.gguf Q4_K_M 4bits 4.37 GB 6.87 GB medium, balanced quality - recommended
Was that a good choice?
Earlier quoted context omitted.
It's very possible to have multiple apps coexisting using a database. Although I'll certainly concede that it's probably a lot easier with just a bunch of Markdown files.
Sure, it's possible, but whichever app owns the database ultimately controls the data, the schema, etc. The file system provides a neutral database that all apps can cooperate within.
Earlier quoted context omitted.
That was the reason why I gave up Joplin very quickly. The last Joplin thread, here on Hacker News, has also shown once again that some still do not understand why "But Joplin can export Markdown from the database!" is not the same as simple, flat Markdown files.
May I ask what you switched to? Running into the same issue.