Show HN: I made an app to use local AI as daily driver • Hacker News Board

Re: Show HN: I made an app to use local AI as daily driver

#31

Post by jlund-molfese » Wed, Feb 28, 2024, 2:42 AM UTC

girishso wrote: post #8

I will totally pay for something like this if it answers from my local documents, bookmarks, browser history etc.

Sounds like https://www.rewind.ai/ ?

Re: Show HN: I made an app to use local AI as daily driver

#32

Post by QuinnyPig » Wed, Feb 28, 2024, 2:47 AM UTC

sandyarmstrong wrote: post #20

christiangenco wrote: post #18

Earlier quoted context omitted.

...how did you highlight a specific sentence like that?

Looks like a Chromium-specific feature: https://web.dev/articles/text-fragments Pretty cool. Doesn't work on Firefox.

It just worked on Safari on iOS. That’s pretty impressive.

Re: Show HN: I made an app to use local AI as daily driver

#33

Post by pentagrama » Wed, Feb 28, 2024, 2:47 AM UTC

Congrats! Plans on Windows support?

Re: Show HN: I made an app to use local AI as daily driver

#34

Post by a_bonobo » Wed, Feb 28, 2024, 3:01 AM UTC

CGamesPlay wrote: post #14

iansinnott wrote: post #9

Earlier quoted context omitted.

You could try out Prompta [1], which I made for this use case. Initially created to use OpenAI as a desktop app, but can use any compatible API including Ollama if you want local completions. [1]: https://github.com/iansinnott/prompta

This one doesn't seem to support system prompts, which are absolutely essential for getting useful output from LLMs.

I've run into the same problem with deploying Gemini locally, it does not seem to support System Prompts. I've cheated around this by auto-prepending the system prompt to the user prompt, and then deleting it from the user-displayed prompt again.

Re: Show HN: I made an app to use local AI as daily driver

#35

Post by giblfiz » Wed, Feb 28, 2024, 3:05 AM UTC

So there are a few questions that leap out at me:

  * What are you using for image generation? Is that local as well (stable diffusion?) Does it have integrated prompt generation? 


  * You mention the ability to import ChatGPT history, are you able to import other documents?

  * How many "agent" style capacities does it have? Can it search the web? use other APIs? Prompt itself? 

  * Does it have a plugin framework? you mention that it is "customizable" but that can mean almost anything. 

  * What is the license? what assurances do users have that their usage is private? I mean, we all know how many "local" apps exfiltrate a ton of data.

Re: Show HN: I made an app to use local AI as daily driver

#36

Post by android521 » Wed, Feb 28, 2024, 3:06 AM UTC

how big is the local model? what is the Mac spec requirement? I don't want to download and find out it won't work in my computer. It seems like the first question everyone would ask and should be addressed on the website.

Re: Show HN: I made an app to use local AI as daily driver

#37

Post by toomuchtodo » Wed, Feb 28, 2024, 3:06 AM UTC

girishso wrote: post #8

I will totally pay for something like this if it answers from my local documents, bookmarks, browser history etc.

https://news.ycombinator.com/item?id=38787892 ("Show HN: Rem: Remember Everything (open source)") ?

https://github.com/jasonjmcghee/rem

Re: Show HN: I made an app to use local AI as daily driver

#38

Post by pentagrama » Wed, Feb 28, 2024, 3:11 AM UTC

Sadly I can't try this because I'm on Windows or Linux.

Was testing apps like this if anyone is interested:

Best / Easy to use:

- https://lmstudio.ai

- https://msty.app

- https://jan.ai

More complex / Unpolished UI:

- https://gpt4all.io

- https://pinokio.computer

- https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat...

- https://github.com/LostRuins/koboldcpp

Misc:

- https://faraday.dev (AI Characters):

No UI / Command line (not for me):

- https://ollama.com

- https://privategpt.dev

- https://serge.chat

- https://github.com/Mozilla-Ocho/llamafile

Pending to check:

- https://recurse.chat

Feel free to recommend more!

Re: Show HN: I made an app to use local AI as daily driver

#39

Post by _boffin_ » Wed, Feb 28, 2024, 3:12 AM UTC

xyc wrote: post #13

girishso wrote: post #8

I will totally pay for something like this if it answers from my local documents, bookmarks, browser history etc.

Yes it would be the next big focus on this. Personal data connectivity is what I see where local AI would excel - despite model power differences.

Good to know there's a market for that. Currently building out something. Integrating from numerous sources, processing and then utilizing those.

nice.

Re: Show HN: I made an app to use local AI as daily driver

#40

Post by chown » Wed, Feb 28, 2024, 3:27 AM UTC

pentagrama wrote: post #38

Sadly I can't try this because I'm on Windows or Linux. Was testing apps like this if anyone is interested: Best / Easy to use: - https://lmstudio.ai - https://msty.app - https://jan.ai More complex / Unpolished UI: - https://gpt4all.io - https://pinokio.computer - https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generat... - https://github.com/LostRuins/koboldcpp Misc: - https://faraday.dev (AI Characters): No U…

I am the author of Msty app mentioned here. So humbled to see an app that is just about a month old that I mostly wrote for my wife and some friends to begin with (who got overwhelmed with everything that was going in LLM world), on the top of your list. Thank you!