Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has Anyone Trained a personal LLM using their personal notes?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 77 posts

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#51

Post by Erazal »

psychopomp wrote: post #18
KMnO4 wrote: post #11

Maybe not exactly what you’re asking, but I started doing talk therapy last year. It’s done virtually and I record the session with OBS. As soon as the recording finishes, the following happens: - The audio is preprocessed (chunked) and sent to Whisper to generate a transcript - The transcript is sent to GPT-4 to generate a summary, action items, concepts introduced with additional information - The next meeting’s da…

I’d love to replicate your workflow. Any luck with speaker diarization using whisper? I’ve tried WhisperX several but it didn’t work.

I've created an AI Meeting Bot API to do just that [1].

At the moment it runs on AWS, and we're thinking of open-sourcing so you could also run it locally to maintain 100% privacy of such conversations.

You'd get speaker diarization, names on top of the recording [2].

[1] https://aimeetingbot.com [2] https://spoke-1.gitbook.io/ai-meeting-bot

Happy to get in touch and have you run it

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#52

Post by Erazal »

cjbprime wrote: post #23

Going to wait for longer context local models. Fine tuning/training is lossy compression of your notes into the model weights -- there isn't much value in a vaguely-remembered copy of some of my notes. This is why other comments are pointing you towards Retrieval Augmented Generation instead, where the relevant notes are losslessly added to the prompt.

I think you're right

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#53

Post by Erazal »

CSMastermind wrote: post #15

It sounds like you want RAG instead of training or even fine tuning a model. Have you looked into the OpenAI APIs? They make it relatively easy to do assuming you have some limited programming knowledge.

I'm currently looking to implement RAG locally, using QDrant [1] for instance.

Just playing around for now, but it makes sense to have a runnable example for our users too :) [2].

[1]. https://qdrant.tech/ [2] https://aimeetingbot.com

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#54

Post by Erazal »

bhupesh wrote: post #44

Folks at gitbook are kind enough to give me a LLM over my notes https://til.bhupesh.me

oh damn, I'm using gitbook too actually - for our users [1]. Not open-source but will definitely try it out ASAP.

[1] https://spoke-1.gitbook.io/ai-meeting-bot

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#57

Post by Erazal »

fullautomation wrote: post #17
KMnO4 wrote: post #11

Maybe not exactly what you’re asking, but I started doing talk therapy last year. It’s done virtually and I record the session with OBS. As soon as the recording finishes, the following happens: - The audio is preprocessed (chunked) and sent to Whisper to generate a transcript - The transcript is sent to GPT-4 to generate a summary, action items, concepts introduced with additional information - The next meeting’s da…

This is really interesting, are your comfortable with OpenAI having your personal details in this case?

This is where having our own LLMs and stacks running locally will save and empower us IMHO

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#59

Post by Erazal »

mitjam wrote: post #40

I have heard good things about the Notion AI addon although I haven’t tried it myself.

have never found it powerful enough - or I'm just setting too high a bar?

I have a ton of databases on Notion (with all my teams conversation transcripts, meeting to-dos, etc.) and global AI search just isn't there.

I haven't found a way there (but have elsewhere using open source) to create a kick-ass search.

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#60

Post by joenot443 »

bredren wrote: post #36

Has anyone seen or used something that can train on a complete imessage history? Presumably, I have more than enough messages from me along with responses from others to chat with a version of myself that bears an incredible likeness to how I speak and think. In some cases, I'd expect to be able to chat with an LLM of a given contact to see how they'd respond to various questions as well.

Came here to ask the same question. I've played around with a few implementations, but nothing with results that were close to useful.

Anyone had success RAG-ing a chat history??

Post reply on HN