This is not local, but uses the Tavily cloud ( https://tavily.com/ ) ?!
Sidekick: Local-first native macOS LLM app
41–50 of 97 posts
Re: Sidekick: Local-first native macOS LLM app
#42The name gave me a flashback to Borland Sidekick
I thought of the phone with the spinning screen from the mid 2000s.
Re: Sidekick: Local-first native macOS LLM app
#43What differentiates this from Open WebUI? How did you design the RAG pipeline?
I had a project in the past where I had hundreds of PDF / HTML files of industry safety and fatality reports which I was hoping to simply "throw in" and use with Open WebUI, but I found it wasn't effective at this even in RAG mode. I wanted to ask it questions like "How many fatalities occurred in 2020 that involved heavy machinery?", but it wasn't able to provide such broad aggregate data.
Re: Sidekick: Local-first native macOS LLM app
#44Neat. It would be nice to provide an option to use an API endpoint without downloading an additional local model. I have several models downloaded via ollama and would prefer to use them without additional space being taken up by the default model.
Re: Sidekick: Local-first native macOS LLM app
#45What differentiates this from Open WebUI? How did you design the RAG pipeline? I had a project in the past where I had hundreds of PDF / HTML files of industry safety and fatality reports which I was hoping to simply "throw in" and use with Open WebUI, but I found it wasn't effective at this even in RAG mode. I wanted to ask it questions like "How many fatalities occurred in 2020 that involved heavy machinery?", but…
I think this is a fundamental issue with naive RAG implementations: they aren't accurate enough for pretty much anything
Re: Sidekick: Local-first native macOS LLM app
#46When I bought my new MBP, I was wondering whether to just upgrade the memory to 48GB thinking that it will become more likely that I will run local models in the next 3-4 year cycle of this laptop. So I took the leap and just upgraded the memory.
Hoping that these kinds of tools will run well in these scenarios.
Re: Sidekick: Local-first native macOS LLM app
#47This needs 164 MB of disk space. Not to bad. Thank you to the author for this.
Re: Sidekick: Local-first native macOS LLM app
#48What an great looking tool. Amazingly generous that it’s open source. Let’s hope the author can keep building it, but if they need to fund their existence there is precedent - lots of folks pay for Superwhisper. People pay for quality software. In a past tech cycle Apple might’ve hired the author, acquired the IP and lovingly stewarded the work into a bundled OS app. Not something to hope for lately. So just going to…
Yep, I think the guy who’ll make the GUI for LLMs is the next Jobs/Gates/Musk and Nobel Prize Winner (I think it’ll solve alignment by having millions of eyes on the internals of LLMs), because computers became popular only after the OS with a GUI appeared. I just started ASK HN to let people and me share their AI safety ideas, both crazy and not: https://news.ycombinator.com/item?id=43332593
What you describe seems to be OpenAI’s “moat”. They currently are the farthest ahead in app UX for nontechnical users and in Brand Recognition. It doesn’t matter if they are 10% behind Anthropic in frontier model quality if Claude For Mac is a shitty Electron app.
Re: Sidekick: Local-first native macOS LLM app
#49What differentiates this from Open WebUI? How did you design the RAG pipeline? I had a project in the past where I had hundreds of PDF / HTML files of industry safety and fatality reports which I was hoping to simply "throw in" and use with Open WebUI, but I found it wasn't effective at this even in RAG mode. I wanted to ask it questions like "How many fatalities occurred in 2020 that involved heavy machinery?", but…
I think this is a fundamental issue with naive RAG implementations: they aren't accurate enough for pretty much anything
Ultimately, the quality of OCR on PDF is where we are bottlenecked as an industry. And not just in text characters but understanding and feeding to the LLM structured object relationships as we see in tables and graphs. Intuitive for a human, very error prone for RAG.
Re: Sidekick: Local-first native macOS LLM app
#50Looks nice, and I greatly appreciate the local only or local first mode. The readme says: > Give the LLM access to your folders, files and websites with just 1 click, allowing them to reply with context. … > Context aware. Aware of your files, folders and content on the web. Am I right in assuming that this works only with local text files and that it cannot integrate with data sources in Apple’s apps such as Notes,…
The Apple data you mention has APIs for feeding them in to LLMs if you wish. Someone just has to write it.
(I wrote one of those Apple API SDKs)