Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has Anyone Trained a personal LLM using their personal notes?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 77 posts

Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#1

Post by Erazal »

Some here comment on notes they've taken for their entire life, often using powerful note-taking systems.

Of course, there's a ton of note-taking systems out there. Org-Mode [1], Obsidian [2], plain .txt, ...

And it's become quite simple to integrate such systems with LLMs.

Whether to add that data to the LLM [3], using LLM formatting, or to visualize and use it as a personal partner. For the latter - there's also a ton of open-source UIs such as Chatbot-ui[4] and Reor[5].

And that's just the tip of the iceberg.

Personally, I haven't been consistent enough through the years in note-taking.

So, I'm really curious to learn more about those of you who were and implemented such pipelines.

I'm sure there's a ton of cool interaction experiences.

[1] https://orgmode.org/ [2] https://obsidian.md/ [3] https://ollama.com/ [4] https://github.com/mckaywrigley/chatbot-ui [5] https://github.com/reorproject/reor

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#4

Post by brotchie »

Played around with fine tuning, but ended up just experimenting with RAG.

One thing I haven’t worked out yet is the agent reliably understanding if it should do a “point retrieval query” or an “aggregation query.”

Point query: embed and do vector lookup with some max N and distance threshold. For example: “Who prepared my 2023 taxes?”

Aggregation query: select a larger collection of documents (1k+) that possibly don’t fit in the context window and reason over the collection. “Summarize all of the correspondence I’ve had with tax preparation agencies over the past 10 years”

The latter may be solved with just a larger max N and larger context window.

Almost like it’s a search lookup vs. a map reduce.

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#5

Post by dleslie »

I want this for my photos.

I'm not a good photographer, but I have taken tens of thousands of photos of my family. I would love to provide a prompt for a specific day and persons and have it create a photo that I never was able to take. I don't mind that it's not "real" because I find photography to be philosophically unreal as it is. I want it to look good, and inspire my mind to recreate the day however it can imagine.

And I want to do it locally, without giving away my family's data and identity.

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#6

Post by Zambyte »

I have a large org-roam note system. I would like to create a pipeline where I can ask natural language questions, and it will build SQLite quries to efficiently crawl through the database and find what I want. I haven't gotten around to it though.

Re: Ask HN: Has Anyone Trained a personal LLM using their personal notes?

#8

Post by dumbmrblah »

I’ve tried several different systems, nothing really stands out.

That being said, I’m trying to document as much as my life in anticipation of such programs existing in the near future. I’m not going overboard, but for example, I wouldn’t really keep a personal diary, but now I try to jot down something every day, write down my thought processes on things, what actions were done and why.

I’m looking forward to a day where I have an AI assistant (locally hosted and under my control of course) who can help me with decision-making based on my previous actions. Would be neat to compare/contrast how I do things now, compared to the future me.

Post reply on HN