Show HN: I made an app to use local AI as daily driver

#1

Post by xyc » Wed, Feb 28, 2024, 12:40 AM UTC

Hi Hackers,

Excited to share a macOS app I've been working on: https://recurse.chat/ for chatting with local AI. While it's amazing that you can run AI models locally quite easily these days (through llama.cpp / llamafile / ollama / llm CLI etc.), I missed feature complete chat interfaces. Tools like LMStudio are super powerful, but there's a learning curve to it. I'd like to hit a middleground of simplicity and customizability for advanced users.

Here's what separates RecurseChat out from similar apps:

- UX designed for you to use local AI as a daily driver. Zero config setup, supports multi-modal chat, chat with multiple models in the same session, link your own gguf file.

- Import ChatGPT history. This is probably my favorite feature. Import your hundreds of messages, search them and even continuing previous chats using local AI offline.

- Full text search. Search for hundreds of messages and see results instantly.

- Private and capable of working completely offline.

Thanks to the amazing work of @ggerganov on llama.cpp which made this possible. If there is anything that you wish to exist in an ideal local AI app, I'd love to hear about it.

Show HN: I made an app to use local AI as daily driver
recurse.chat

Re: Show HN: I made an app to use local AI as daily driver

#2

Post by CGamesPlay » Wed, Feb 28, 2024, 1:26 AM UTC

Possibly a strange question, but do you have plans to add online models to the app? Local models just aren't at the same level, but I would certainly appreciate a consistent chat interface that lets me switch between GPT/Claude/local models.

Re: Show HN: I made an app to use local AI as daily driver

#3

Post by raajg » Wed, Feb 28, 2024, 1:27 AM UTC

looks promising, but after looking at the website I'm yearning to learn more about it! How does it compare to alternatives? What's the performance like? There isn't enough to push me to stop using ChatGPT and use this instead. Offline is good, but to get users at scale there has to be a compelling reason to shift. I don't think that offline capabilities are going to be enough to get significant number of users.

Another tip, I try out a new chat interface to LLMs almost every week and they're free to use initially. There isn't a compelling reason for me to spend $10 from the get to for a use case that I'm not sure about yet.

Re: Show HN: I made an app to use local AI as daily driver

#4

Post by tkgally » Wed, Feb 28, 2024, 1:30 AM UTC

For an app like this, I would really like a spoken interface. Any possibility of adding text-to-speech and speech-to-text so that users can not only type but also talk with it?

Re: Show HN: I made an app to use local AI as daily driver

#5

Post by [deleted] » Wed, Feb 28, 2024, 1:32 AM UTC

CGamesPlay wrote: post #2

Possibly a strange question, but do you have plans to add online models to the app? Local models just aren't at the same level, but I would certainly appreciate a consistent chat interface that lets me switch between GPT/Claude/local models.

[deleted]

Re: Show HN: I made an app to use local AI as daily driver

#6

Post by xyc » Wed, Feb 28, 2024, 1:32 AM UTC

CGamesPlay wrote: post #2

Possibly a strange question, but do you have plans to add online models to the app? Local models just aren't at the same level, but I would certainly appreciate a consistent chat interface that lets me switch between GPT/Claude/local models.

Not strange at all! It's a very valid ask. The focus is local AI, but GPT-3.5/GPT-4 are actually included in the app (bring your own key), although customization is limited. Planning to expose some more customizability there including API base urls / model names.

Re: Show HN: I made an app to use local AI as daily driver

#7

Post by castles » Wed, Feb 28, 2024, 1:32 AM UTC

CGamesPlay wrote: post #2

Possibly a strange question, but do you have plans to add online models to the app? Local models just aren't at the same level, but I would certainly appreciate a consistent chat interface that lets me switch between GPT/Claude/local models.

https://recurse.chat/faq/#:~:text=We%20support%20Mistral%2C%...

Re: Show HN: I made an app to use local AI as daily driver

#8

Post by girishso » Wed, Feb 28, 2024, 1:35 AM UTC

I will totally pay for something like this if it answers from my local documents, bookmarks, browser history etc.

Re: Show HN: I made an app to use local AI as daily driver

#9

Post by iansinnott » Wed, Feb 28, 2024, 1:37 AM UTC

CGamesPlay wrote: post #2

Possibly a strange question, but do you have plans to add online models to the app? Local models just aren't at the same level, but I would certainly appreciate a consistent chat interface that lets me switch between GPT/Claude/local models.

You could try out Prompta [1], which I made for this use case. Initially created to use OpenAI as a desktop app, but can use any compatible API including Ollama if you want local completions.

[1]: https://github.com/iansinnott/prompta

Re: Show HN: I made an app to use local AI as daily driver

#10

Post by rbtprograms » Wed, Feb 28, 2024, 1:37 AM UTC

Looks great! Does it support different sized models, i.e. can I run llama 70B and 7B, and is there a way to specify which model to chat with? Are there plans to allow users to ingest their own models through this UI?