Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I made an app to use local AI as daily driver

recurse.chat

Post reply on HN
Print view View original

101–110 of 246 posts

Re: Show HN: I made an app to use local AI as daily driver

#101

Post by woadwarrior01 »

pentagrama wrote: post #38

Sadly I can't try this because I'm on Windows or Linux. Was testing apps like this if anyone is interested: Best / Easy to use: - https://lmstudio.ai - https://msty.app - https://jan.ai More complex / Unpolished UI: - https://gpt4all.io - https://pinokio.computer - https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat... - https://github.com/LostRuins/koboldcpp Misc: - https://faraday.dev (AI Characters): No U…

Since I couldn't find it in your list, I'd like to plug my own macOS (and iOS) app: Private LLM. Unlike almost every other app in the space, it isn't based on llama.cpp (we use mlc-llm) or naive RTN quantized models (we use OmniQuant). Also, the app has deep integrations with macOS and iOS (Shortcuts, Siri, macOS Services, etc).

Incidentally, it currently runs Mixtral 8x7B Instruct[2] and Mistral[3] models faster than any other macOS app. The comparison videos are with Ollama, but it generalizes well to almost every other macOS app that I've seen uses llama.cpp for inference. :)

nb: Mixtral 8x7B Instruct requires an Apple Silicon Mac with at least 32GB of RAM.

[1]: https://privatellm.app/

[2]: https://www.youtube.com/watch?v=CdbxM3rkxtc

[3]: https://www.youtube.com/watch?v=UIKOjE9NJU4

Re: Show HN: I made an app to use local AI as daily driver

#103

Post by greggsy »

joshmarinacci wrote: post #57
pentagrama wrote: post #38

Sadly I can't try this because I'm on Windows or Linux. Was testing apps like this if anyone is interested: Best / Easy to use: - https://lmstudio.ai - https://msty.app - https://jan.ai More complex / Unpolished UI: - https://gpt4all.io - https://pinokio.computer - https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat... - https://github.com/LostRuins/koboldcpp Misc: - https://faraday.dev (AI Characters): No U…

Do any of these let you dump in a bunch of your own documents to use as a corpus and then query and summarize them ?

https://github.com/imartinez/privateGPT

Re: Show HN: I made an app to use local AI as daily driver

#105

Post by AriedK »

chown wrote: post #58
lolpanda wrote: post #52

Earlier quoted context omitted.

Oh thanks! didn't know there are quite a few ChatGPT local alternatives. I was wondering what users they are targeting. Engineers or average users? I guess average users will likely choose ChatGPT and Perplexity over local apps for more recent knowledge of the world.

Hi. I'm the author of Msty app, 2nd on the list above. You are right about average users likely choosing ChatGPT over local models. My wife was the first and the biggest user of my app. A software engineer by profession and training but she likes to not worry about LLM world and just to use it as a tool that makes you more productive. As soon as she took Msty for a ride, I realized that some users, despite their back…

Looks great, though the fact that you have to ignore your anti-virus warning during installation, and the fact that it phones home (to insights.msty.app) directly after launch despite the line in the FAQ on not collecting any data makes me a little skittish.

Re: Show HN: I made an app to use local AI as daily driver

#106

Post by tethys »

spiderfarmer wrote: post #102
girishso wrote: post #8

I will totally pay for something like this if it answers from my local documents, bookmarks, browser history etc.

Next version of MacOS will probably have that.

As long as you use Safari for browsing, Notes for note taking, iCloud for mail …

Re: Show HN: I made an app to use local AI as daily driver

#107

Post by crooked-v »

chown wrote: post #40
pentagrama wrote: post #38

Sadly I can't try this because I'm on Windows or Linux. Was testing apps like this if anyone is interested: Best / Easy to use: - https://lmstudio.ai - https://msty.app - https://jan.ai More complex / Unpolished UI: - https://gpt4all.io - https://pinokio.computer - https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat... - https://github.com/LostRuins/koboldcpp Misc: - https://faraday.dev (AI Characters): No U…

I am the author of Msty app mentioned here. So humbled to see an app that is just about a month old that I mostly wrote for my wife and some friends to begin with (who got overwhelmed with everything that was going in LLM world), on the top of your list. Thank you!

One bit of feedback: there's nowhere to put system messages. These can be much more influential than user prompts when it comes to shaping the tone and style of the response.

Re: Show HN: I made an app to use local AI as daily driver

#108

Post by longnguyen »

CGamesPlay wrote: post #2

Possibly a strange question, but do you have plans to add online models to the app? Local models just aren't at the same level, but I would certainly appreciate a consistent chat interface that lets me switch between GPT/Claude/local models.

Shameless plug: if you need multiple AI Service Provider, give BoltAI[0] a try. It’s native (not Electron), and supports multiple services: OpenAI, Azure OpenAI, OpenRouter, Mistral, Ollama…

It also allows you to interact with LLMs via multiple different interfaces: Chat UI, a context-aware called AI Command and an Inline mode.

[0]: https://boltai.com

Re: Show HN: I made an app to use local AI as daily driver

#109

Post by pantulis »

xyc wrote: post #71
rexreed wrote: post #30

What are the MacOS and hardware requirements? How does it perform on a slightly older model, lower powered Mac? I wish I could test this to see how it would perform, and while it's only $10, I don't want to spend that just to realize it won't work on my older, underpowered Mac mini.

Good question, I'll put some system requirements on the website. It only supports mac with Apple Silicon now, if that's helpful.

Instant buy, great work and the price point is exactly right. Good luck!

Re: Show HN: I made an app to use local AI as daily driver

#110

Post by sigmoid10 »

woadwarrior01 wrote: post #101
pentagrama wrote: post #38

Sadly I can't try this because I'm on Windows or Linux. Was testing apps like this if anyone is interested: Best / Easy to use: - https://lmstudio.ai - https://msty.app - https://jan.ai More complex / Unpolished UI: - https://gpt4all.io - https://pinokio.computer - https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat... - https://github.com/LostRuins/koboldcpp Misc: - https://faraday.dev (AI Characters): No U…

Since I couldn't find it in your list, I'd like to plug my own macOS (and iOS) app: Private LLM. Unlike almost every other app in the space, it isn't based on llama.cpp (we use mlc-llm) or naive RTN quantized models (we use OmniQuant). Also, the app has deep integrations with macOS and iOS (Shortcuts, Siri, macOS Services, etc). Incidentally, it currently runs Mixtral 8x7B Instruct[2] and Mistral[3] models faster tha…

What's the performance like in tokens/s?
Post reply on HN