Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I made an app to use local AI as daily driver

recurse.chat

Post reply on HN
Print view View original

221–230 of 246 posts

Re: Show HN: I made an app to use local AI as daily driver

#221

Post by hanniabu »

chown wrote: post #40
pentagrama wrote: post #38

Sadly I can't try this because I'm on Windows or Linux. Was testing apps like this if anyone is interested: Best / Easy to use: - https://lmstudio.ai - https://msty.app - https://jan.ai More complex / Unpolished UI: - https://gpt4all.io - https://pinokio.computer - https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat... - https://github.com/LostRuins/koboldcpp Misc: - https://faraday.dev (AI Characters): No U…

I am the author of Msty app mentioned here. So humbled to see an app that is just about a month old that I mostly wrote for my wife and some friends to begin with (who got overwhelmed with everything that was going in LLM world), on the top of your list. Thank you!

1) What are the mac system requirements? Does it need a specific OS version?

2) If you're privacy first, many would feel a lot more comfortable if this was released as an app in the app store so it will be sandboxed. This is important because it's not open source so we have no idea what is happening in the background. Alternatively open source it, which many here have requested.

Re: Show HN: I made an app to use local AI as daily driver

#222

Post by appel »

xyc wrote: post #74
pentagrama wrote: post #33

Congrats! Plans on Windows support?

Thanks! Sorry no immediate plan. People have recommended Chat with RTX so it might be worth checking out. https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat...

It looks amazing, OP! I'm sad I'm missing out as a Windows user.

Re: Show HN: I made an app to use local AI as daily driver

#224

Post by Satam »

akerl_ wrote: post #211
Satam wrote: post #138

Earlier quoted context omitted.

To use Google's sensitive APIs in production you have to certify your product and that costs tens of thousands. To be honest, didn't think about imap at first, but it looks like that could be getting tougher soon too https://support.google.com/a/answer/14114704?hl=en . Soon they will require oAuth for imap and with oAuth you'll need the certification: https://developers.google.com/gmail/imap/xoauth2-protocol . If it'…

I'm not sure what you mean. > You will no longer use a password for access (with the exception of app passwords) I'm not seeing anywhere that I'd need to pay money to use OAuth via an app like Thunderbird or another email client. That app would either need to support using OAuth to let the user auth and get credentials, or use an app password.

Right, but Thunderbird had to pay up and set themselves as a middleman to allow this. My point is that local LLMs might not have that many advantages for personal data because most of that data doesn't live locally on your computer, to begin with. I guess an argument could be made that running them locally prevents an AI provider from gobbling up ALL of your data. On the other hand, Google already has most of our my data: emails, youtube, gmail, etc.

Re: Show HN: I made an app to use local AI as daily driver

#225

Post by Grimblewald »

tartrate wrote: post #117

> Full Text Search. Blazingly fast search over thousands of messages. Natural language processing has come full circle and just reinvented Ctrl+F. I had to double check that a regular '90s search function was actually the thing being advertised here, and sure enough, there is a gif demonstrating exactly that.

I'm a big fan of ctrl+f, but semantic search is a life saver that conventional search simply cannot compare to.

Re: Show HN: I made an app to use local AI as daily driver

#226

Post by Grimblewald »

danielovichdk wrote: post #149
onehp wrote: post #145

Earlier quoted context omitted.

Retrieval augmented generation. In short you use an LLM to classify your documents (or chunks from them) up front. Then when you want to ask the LLM a question you pull the most relevant ones back to feed it as additional context.

I dont get it. To my understanding it takes huge amounts of data to build any any form of RAG. Simply because it enlarges the statistical model you later prompt. If the model is not big enough how would you expect it to answer you in a non qualifying matter ? It simply can't. So I don't really buy it and I have yet to see it work better than any rdbms search index. Tell me I am wrong, I would like to see a local mode…

You're misunderstanding. Imagine your query is matched against chunks of text from the database, where the relevance of information is evaluated for the window each time it slides. Then collecting the n most relevant chunks, these are included in the prompt so then the llm can provide answers from source documents verbatim. This is useful for cases where precise and exact answers are needed. For example, searching the docs for some package for the right api to call. You don't mant a name that's close to right it has to be correct to the character.

Re: Show HN: I made an app to use local AI as daily driver

#227

Post by wkat4242 »

onehp wrote: post #145
wkat4242 wrote: post #142

Earlier quoted context omitted.

Stupid question but what does RAG stand for?

Retrieval augmented generation. In short you use an LLM to classify your documents (or chunks from them) up front. Then when you want to ask the LLM a question you pull the most relevant ones back to feed it as additional context.

Ahh ok I see. It's basically what MS CoPilot 365 does too, with its "Grounding" step.

Re: Show HN: I made an app to use local AI as daily driver

#228

Post by toomuchtodo »

gabev wrote: post #208
samstave wrote: post #199

Earlier quoted context omitted.

Seems like this would be a good tool to build lessons on - if you could share a "class" and export a link for others to then copy the class, and expand on the lesson/class/topic into their own AI. but as a separate "class" and not fully integrated to my regular history blob? I want the ability to search all my downloaded files and organize them based on context within. Have it create a category table, and allow me to…

We're working on the ability to share folders of your knowledge so that others can search/chat across them. We've been thinking of this as a "subscription" to the creator's folder. Similar to how you might subscribe to a Spotify playlist

Consider using tar files for this. Lots of tooling (versioning, hashing, storage) around this already, and docker layers comes to mind.

Re: Show HN: I made an app to use local AI as daily driver

#229

Post by toomuchtodo »

wodow wrote: post #200
ApolloFortyNine wrote: post #196

Earlier quoted context omitted.

I have ChatGPT4, I have no idea what arrow you are talking about. Could you be more specific? I see now arrow on any of my previous messages or current ones.

By George, ItsMattyG is right! After editing a question (with the "stylus"/pen icon), the revision number counter that appears (e.g. "1 / 2") has arrows next to it that allow forward and backward navigation through the new branches. This was surprisingly undiscoverable. I wonder if it's documented. I couldn't find anything from a quick look at help.openai.com .

Careful what you trust with help.openai.com. You used to be able to share conversations, now it's login walled when you share, and the docs don't reflect this (if someone can recommend a frontend that has this functionality, for quick sharing of conversations with others via a link, taking recommendations, thank you in advance).

Re: Show HN: I made an app to use local AI as daily driver

#230

Post by xyc »

howmayiannoyyou wrote: post #173

Without Apple Shortcuts support I can't pay for this. I get pretty much the same experience from GPT4All. Hoping you add support CLI, Shortcuts or something along those lines.

Thanks for the suggestion! I play with Apple Shortcuts sometimes. It's an exemplary example of how easy end user programming could be. Will keep this in mind.
Post reply on HN