Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has Anyone Trained a personal LLM using their personal notes?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 77 posts

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#41

Post by mewpmewp2 »

melagonster wrote: post #38
serial_dev wrote: post #37

Earlier quoted context omitted.

English is not my first language, so I beg for your forgiveness that I didn't use the they/them. In my language, there is no gendered pronouns, so it's hard for me to guess what's the correct way. When I use he/him, people tell me I assume everyone on HN and tech is a male. When I use she/her, you come and label me as misogynistic, too. At one point I just give up. Or you know, you could also assume positive intent a…

you can use "they" at every where today. but I don't whether someone will accuse you support Trans community.

Another possibility is to use "op", although it can feel a bit forumish.

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#42

Post by appstorelottery »

I've been disciplined (perhaps obsessive at times) with keeping a daily diary for many years and I was interested in being able to query my diary locally via AI. I found a solution that works surprisingly well using GPT4ALL.

I found GPT4ALL (https://gpt4all.io) to have a nice-enough GUI, and it runs reasonably quickly on my M1 MacBook Air with 8Gb of ram, and it can be setup to be a completely local solution - not sending your data to the Goliaths.

GPT4ALL has an option to access local documents, via the Sbert text embedding model (RAG).

My specific results have been as follows; using the Nous Hermes 2 Mistral DPO and Sbert - I indexed 153 days of my daily writing (most days I write between 2 and 3 thousand words).

Asking a simple question like "what are the challenges faced by the author?" provides remarkable, almost spooky results (which I won't share here) - which in my opinion are spot-on regarding my own challenges over that the period - and Sbert provides references to which documents it used to generate the answer. Options are available to reference an arbitrary number of documents, however the default is 10. Ideally I'd like to have it reference all 153 documents in the query - I'm not sure if it's a ram or a token issue, however increasing the value of documents referenced has resulted in machine lock-ups.

Anyhow - that's my experience - hope it's helpful to someone.

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#43

Post by aaron695 »

mattlondon wrote: post #31
serial_dev wrote: post #29

Earlier quoted context omitted.

I'm sure OP knows this, thought about the risk, guessed the probabilities, assessed the nature of her therapy sessions, and decided the system she was building is worth the risk of her therapy content being available with an intuitive chat interface. Not every therapy session is embarrassing, incriminating, or particularly interesting to anyone but you.

Off topic,but where did that gendering come from in your response? I am generally indifferent to personal pronouns but somehow this came across as particularly jarring as I guess it felt like a projection of misogynistic views of female weakness/need for therapy/emotionalness/etc. They/their might have been more appropriate here.

[dead]

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#45

Post by serial_dev »

mewpmewp2 wrote: post #41
melagonster wrote: post #38

Earlier quoted context omitted.

you can use "they" at every where today. but I don't whether someone will accuse you support Trans community.

Another possibility is to use "op", although it can feel a bit forumish.

I used OP, though, I didn't want to use "OP" every time I am referencing this person, I thought it would sound strange OP OP OP...

Anyway, I'm glad this thread turned into a "how you can avoid being called a misogynist in throwaway comment on HN". I guess from now on, I filter through my comments through Gemini to make sure my English is immaculate and that native English speakers don't castigate me for a small mistake.

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#46

Post by injuly »

appstorelottery wrote: post #42

I've been disciplined (perhaps obsessive at times) with keeping a daily diary for many years and I was interested in being able to query my diary locally via AI. I found a solution that works surprisingly well using GPT4ALL. I found GPT4ALL ( https://gpt4all.io ) to have a nice-enough GUI, and it runs reasonably quickly on my M1 MacBook Air with 8Gb of ram, and it can be setup to be a completely local solution - not…

Is the diary digital? I prefer writing on paper, and I'd like to try this. Wonder if there's any decent OCR app that'll help me do it.

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#47

Post by resource_waste »

appstorelottery wrote: post #42

I've been disciplined (perhaps obsessive at times) with keeping a daily diary for many years and I was interested in being able to query my diary locally via AI. I found a solution that works surprisingly well using GPT4ALL. I found GPT4ALL ( https://gpt4all.io ) to have a nice-enough GUI, and it runs reasonably quickly on my M1 MacBook Air with 8Gb of ram, and it can be setup to be a completely local solution - not…

This might seem impressive because of the subjectiveness. I also imagine, you arent mentioning the times it was completely incorrect because you used a negative in the sentence.

This is regular embeddings + LLM.

At the end of the day, you are basically just adding a preprompt to a search. Not to mention, the Mistral models are barely useful for logic.

I'm not really sure what you are getting out of it. I'm wondering if you are reading some mostly generic Mistral output with a few words from your pre-prompt/embedding.

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#48

Post by aerhardt »

mmahemoff wrote: post #34
kid64 wrote: post #12

Earlier quoted context omitted.

I hope you're the patient in this scenario, otherwise this is an egregious HIPAA violation.

Might still be a violation if they're the patient? Unless therapist and their employer's consent is given and ofc dependent on the relevant jurisdictions (IANAL).

I don’t know about the legality of it, but as a comical skit it’d be hilarious: a patient gets in deep shit with their doctor for violating patient-doctor confidentiality. Sounds straight out of Curb Your Enthusiasm!

Edit: It seems it's straight out of Curb, because it is! https://youtu.be/YH55dFlF_Rg?si=kOLC5rGq5fi8tke2

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#49

Post by 2rsf »

jki275 wrote: post #2

PrivateGPT is a nice tool for this. It's not exactly what you're asking for, but it gets part of the way there. https://github.com/zylon-ai/private-gpt

I used Private GPT with our internal markdown based developers portal, the results are OK-ish but are closer to a fancy search then to a chat.

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#50

Post by Erazal »

KMnO4 wrote: post #11

Maybe not exactly what you’re asking, but I started doing talk therapy last year. It’s done virtually and I record the session with OBS. As soon as the recording finishes, the following happens: - The audio is preprocessed (chunked) and sent to Whisper to generate a transcript - The transcript is sent to GPT-4 to generate a summary, action items, concepts introduced with additional information - The next meeting’s da…

wow - really cool.

I'm actually the founder of an AI Meeting Bot company - and we're thinking of open-sourcing so you could run exactly this set-up locally with perfect diarization / recording while also maintaining privacy [1].

I'm currently creating code examples, and just finished the "chat with each session". Would love to know how you implemented it.

[1] https://aimeetingbot.com

Post reply on HN