Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I made an app to use local AI as daily driver

recurse.chat

Post reply on HN
Print view View original

231–240 of 246 posts

Re: Show HN: I made an app to use local AI as daily driver

#231

Post by Moldoteck »

tartrate wrote: post #117

> Full Text Search. Blazingly fast search over thousands of messages. Natural language processing has come full circle and just reinvented Ctrl+F. I had to double check that a regular '90s search function was actually the thing being advertised here, and sure enough, there is a gif demonstrating exactly that.

i'm more interested how to perform semantic search over messages efficiently. Like to receive the reference to the og message. Is it creating an llm response with the potential content and how does it find the og message? is it performing tf/idf+cosine search after that or how?

Re: Show HN: I made an app to use local AI as daily driver

#232

Post by brianjking »

wkat4242 wrote: post #227
onehp wrote: post #145

Earlier quoted context omitted.

Retrieval augmented generation. In short you use an LLM to classify your documents (or chunks from them) up front. Then when you want to ask the LLM a question you pull the most relevant ones back to feed it as additional context.

Ahh ok I see. It's basically what MS CoPilot 365 does too, with its "Grounding" step.

Yes.

Re: Show HN: I made an app to use local AI as daily driver

#233

Post by replwoacause »

tartrate wrote: post #117

> Full Text Search. Blazingly fast search over thousands of messages. Natural language processing has come full circle and just reinvented Ctrl+F. I had to double check that a regular '90s search function was actually the thing being advertised here, and sure enough, there is a gif demonstrating exactly that.

Yes but this is missing from most chatbot UIs (ChatGPT, Gemini, Claude, etc. ) so is therefore very useful. Machato has it but it is very laggy.

Re: Show HN: I made an app to use local AI as daily driver

#234

Post by sneak »

You can’t buy these apps from the Apple store without providing identity to Apple in the form of a phone number (required for an Apple ID) and linking it to a hardware serial number (the app store app transmits this).

It would be nice to be able to buy the app directly from you, instead of putting a surveillance company in the loop. I don’t use an Apple ID on a macintosh.

(I would like to avoid using one on an iPhone/iPad/Vision Pro/AppleTV, but it is impossible to install apps on those without one. Please do not bring this terrible circumstance to be the default on macs, too.)

Re: Show HN: I made an app to use local AI as daily driver

#235

Post by lee-rhapsody »

danielovichdk wrote: post #151
addandsubtract wrote: post #119

Earlier quoted context omitted.

Ctrl+F only gets you so far. It doesn't allow you to perform semantic searches, for example. If you don't happen to know a unique word (or set of words) to search for, you're out of luck. Just the other day, I was able to find a song by typing the phonetic pronunciation (well, as best I could) into ChatGPT, and it knew which song I was talking about right away. No way a regular search engine would've helped me there.

No. Your own data only gets you so far. And this is exactly the issue. No local model will make sense because the dataset its given is so small compared to what you are referring to - chatgpt. It's useless locally.

If you only need to query your own data, and can't upload that data to ChatGPT for compliance/security reasons, I think local LLMs are far from useless.

Re: Show HN: I made an app to use local AI as daily driver

#236

Post by djfdat »

copperx wrote: post #98
vunderba wrote: post #55

Earlier quoted context omitted.

If your ultimate goal is privacy, then you should only be looking at open source chat UI front ends: https://github.com/mckaywrigley/chatbot-ui https://github.com/oobabooga/text-generation-webui https://github.com/mudler/LocalAI And then connecting them to off-line models servers: - Ollama - llama.cpp And you should avoid closed source frontends: - Recurse - LM Studio And closed source models - ChatGPT - Gemini

Are you implying Claude is an open source model?

I don't think the list was meant to be exhaustive.

Re: Show HN: I made an app to use local AI as daily driver

#237

Post by antifa »

davely wrote: post #204
tartrate wrote: post #117

> Full Text Search. Blazingly fast search over thousands of messages. Natural language processing has come full circle and just reinvented Ctrl+F. I had to double check that a regular '90s search function was actually the thing being advertised here, and sure enough, there is a gif demonstrating exactly that.

Yeah, I think the call out here is specifically because you the ChatGPT interface doesn't have a search feature (on web). Interestingly, on their iOS app, you can search. I often find myself opening the app on my phone if I want to find a previous conversation, even if I'm at my desk.

The search feature on ChatGPT for Android only works for a tiny number of recent chats.

Re: Show HN: I made an app to use local AI as daily driver

#238

Post by bigfudge »

eevmanu wrote: post #217
bigfudge wrote: post #175

Earlier quoted context omitted.

Although RAG is often implemented via vector databases to find 'relevant' content, I'm not sure that's a necessary component. I've been doing what I call RAG by finding 'relevant' content for the current prompt context via a number of different algorithms that don't use vectors. Would you define RAG only as 'prompt optimisation that involves embeddings'?

Sure thing, your RAG approach sounds intriguing, especially since you're sidestepping vector databases. But doesn't the input context length cap affect it? (chatgpt plus at 32K [0] or gpt4 via open ai at 128K [1]) Seems like those cases would be pretty rare though. [0]: https://openai.com/chatgpt/pricing#:~:text=8K-,32K,-32K [1]: https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turb...

Yes, context window is a limiting factor, but that's true however you identify the content to augment generation.

Re: Show HN: I made an app to use local AI as daily driver

#239

Post by python273 »

wodow wrote: post #165
domano wrote: post #163

Hey, i bought it, nice work! A few things: * The main thing that makes ChatGPTs ui useful to me is the ability to change any of my prompts in the conversation & it will then go back to that part of the converation and regenerate, while removing the rest of the conversation after that point. Such a chat ui is not usable for me without this feature. * The feedback button does nothing for me, just changes focus to chrom…

> * The main thing that makes ChatGPTs ui useful to me is the ability to change any of my prompts in the conversation & it will then go back to that part of the converation and regenerate, while removing the rest of the conversation after that point. Agreed, but what I would also really like (from this and ChatGPT) would be branching: take a conversation in two different ways from some point and retain the seperate a…

I have a very simple UI with threading. It's really unpolished though.

https://eimi.cns.wtf/

https://github.com/python273/eimi

Post reply on HN