Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I made an app to use local AI as daily driver

recurse.chat

Post reply on HN
Print view View original

141–150 of 246 posts

Re: Show HN: I made an app to use local AI as daily driver

#141

Post by quickthrower2 »

pentagrama wrote: post #38

Sadly I can't try this because I'm on Windows or Linux. Was testing apps like this if anyone is interested: Best / Easy to use: - https://lmstudio.ai - https://msty.app - https://jan.ai More complex / Unpolished UI: - https://gpt4all.io - https://pinokio.computer - https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat... - https://github.com/LostRuins/koboldcpp Misc: - https://faraday.dev (AI Characters): No U…

Thanks for the list. Tried Jan just now as it is both easy and open source. It is a bit buggy I think but the concept is ace. The quick install, tells you which models work on your machine, one click download and then a chatgpt style interface. Mistral 7B running on my low spec laptop at 6 token/s making some damn sense is amazing. The bugs are at the inference time. Could be hardware issues though, not sure. YMMV

Re: Show HN: I made an app to use local AI as daily driver

#142

Post by wkat4242 »

vunderba wrote: post #56
girishso wrote: post #8

I will totally pay for something like this if it answers from my local documents, bookmarks, browser history etc.

There are already several RAG chat open source solutions available. Two that immediately come to mind are: Danswer https://github.com/danswer-ai/danswer Khoj https://github.com/khoj-ai/khoj

Stupid question but what does RAG stand for?

Re: Show HN: I made an app to use local AI as daily driver

#143

Post by _ink_ »

rkuodys wrote: post #95

Honest question - can it be used for programming? Or anyone maybe can recommend local-first development LLM which would take in all project (Python / Angular) and write code based on full repo, not only the active window as with Copilot or Jetbrains AI

Check out the continue dev plugin (available for VS Code and Jetbrains). You can attach it to OpenAI or local models and it can consider files in your codebase. It has a @Codebase keyword, but so far I get better results in specifically pointing to the needed files.

Re: Show HN: I made an app to use local AI as daily driver

#144

Post by indit »

chown wrote: post #68
devinprater wrote: post #62

There's another one someone made for blind users like themselves and me, called Vollama (they use a mac, so VoiceOver + Llama). It's really good. I haven't tested many others for accessibility, but it has RAG and uses Ollama as backend, so works very well for me. https://github.com/chigkim/VOLlama/

It's very nice that there exists something like that. I am an author of one of the similar apps [1] someone listed in a different thread. I was hoping I could get in touch with someone like you who could give me some feedback on how to make my app more accessible for users like you. I really want to it be an "LLM for all" kind of app but despite my best efforts and intention, I suck at it. Any chance of getting in to…

Hi, I just use msty. Could it use an already downloaded gguf file?

Re: Show HN: I made an app to use local AI as daily driver

#145

Post by onehp »

wkat4242 wrote: post #142
vunderba wrote: post #56

Earlier quoted context omitted.

There are already several RAG chat open source solutions available. Two that immediately come to mind are: Danswer https://github.com/danswer-ai/danswer Khoj https://github.com/khoj-ai/khoj

Stupid question but what does RAG stand for?

Retrieval augmented generation. In short you use an LLM to classify your documents (or chunks from them) up front. Then when you want to ask the LLM a question you pull the most relevant ones back to feed it as additional context.

Re: Show HN: I made an app to use local AI as daily driver

#146

Post by noduerme »

Satam wrote: post #138
noduerme wrote: post #128

Earlier quoted context omitted.

Seconding a sibling question: What $50k API fee? To access your gmail? I've been using gmail since 2008 or so without ever touching their web/app interface or getting an API key. You just use it as an IMAP server.

To use Google's sensitive APIs in production you have to certify your product and that costs tens of thousands. To be honest, didn't think about imap at first, but it looks like that could be getting tougher soon too https://support.google.com/a/answer/14114704?hl=en . Soon they will require oAuth for imap and with oAuth you'll need the certification: https://developers.google.com/gmail/imap/xoauth2-protocol . If it'…

Yeah, Thunderbird integrated oAuth in the last few releases, mainly to keep up with the Gmail and Hotmail requirements. Made it very user-friendly to set up in the GUI right within T-bird. I don't see this being a major obstacle.

I'm not sure I can imagine a scenario in production where Google would, or should, allow API access to individual gmail accounts. What's that for? So you can read all your employees' mail without running your own email server?

Re: Show HN: I made an app to use local AI as daily driver

#147

Post by jiriro »

rangera wrote: post #132
jiriro wrote: post #124

Out of curiosity – how is this app built?:-) There is a demo clip with a vertical scroll bar which does not fade out as it would do in a native mac app:)

Scroll bars don't fade out if you're using a mouse (as opposed to just a trackpad) or if you've set Mac OS Settings > Appearance > Show scroll bars to "Always".

I see! I’ve not used mouse on a mac:-o

Anyway the UI looks not mac native. I’m interested what it is:-)

Re: Show HN: I made an app to use local AI as daily driver

#148

Post by smnscu »

pentagrama wrote: post #38

Sadly I can't try this because I'm on Windows or Linux. Was testing apps like this if anyone is interested: Best / Easy to use: - https://lmstudio.ai - https://msty.app - https://jan.ai More complex / Unpolished UI: - https://gpt4all.io - https://pinokio.computer - https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat... - https://github.com/LostRuins/koboldcpp Misc: - https://faraday.dev (AI Characters): No U…

Nice, adding these to my list. Here's a list that I put together, it has active GitHub projects for LLM UIs, ordered by stars:

- https://github.com/nomic-ai/gpt4all

- https://github.com/imartinez/privateGPT

- https://github.com/oobabooga/text-generation-webui

- https://github.com/FlowiseAI/Flowise

- https://github.com/lobehub/lobe-chat

- https://github.com/PromtEngineer/localGPT

- https://github.com/h2oai/h2ogpt

- https://github.com/huggingface/chat-ui

- https://github.com/SillyTavern/SillyTavern

- https://github.com/ollama-webui/ollama-webui

- https://github.com/Chainlit/chainlit

- https://github.com/LostRuins/koboldcpp

- https://github.com/ParisNeo/lollms-webui/

Re: Show HN: I made an app to use local AI as daily driver

#149

Post by danielovichdk »

onehp wrote: post #145
wkat4242 wrote: post #142

Earlier quoted context omitted.

Stupid question but what does RAG stand for?

Retrieval augmented generation. In short you use an LLM to classify your documents (or chunks from them) up front. Then when you want to ask the LLM a question you pull the most relevant ones back to feed it as additional context.

I dont get it. To my understanding it takes huge amounts of data to build any any form of RAG. Simply because it enlarges the statistical model you later prompt. If the model is not big enough how would you expect it to answer you in a non qualifying matter ? It simply can't.

So I don't really buy it and I have yet to see it work better than any rdbms search index.

Tell me I am wrong, I would like to see a local model based on my own docs being able to answer me quality answers based on quality prompts.

Re: Show HN: I made an app to use local AI as daily driver

#150

Post by gverrilla »

bradnickel wrote: post #47
raajg wrote: post #3

looks promising, but after looking at the website I'm yearning to learn more about it! How does it compare to alternatives? What's the performance like? There isn't enough to push me to stop using ChatGPT and use this instead. Offline is good, but to get users at scale there has to be a compelling reason to shift. I don't think that offline capabilities are going to be enough to get significant number of users. Anoth…

The compelling reason to shift to local/decentralized AI is that all of compute will soon be AI and that means your entire existence will go into it. The question you should ask yourself is do you want everything about you being handled by Sam Altman, Google, Microsoft, etc? Do you want all of your compute dependent on them always being up and do you want to trust their security team with your life? Do you want to st…

> when truly open AI surpasses all of them in performance and capability.

Is this true? I've tried llama last year and it was not very helpful. GPT4 is already full of problems and I have to keep circumventing them, so using something less capable doesn't get me too excited.

Post reply on HN