I used GPT to build a search tool for my second brain note-taking system
reasonabledeviations.com
I used GPT to build a search tool for my second brain note-taking system
1–10 of 103 posts
Re: I used GPT to build a search tool for my second brain note-taking system
#2Would be nice to see some indication of how well it works in his case.
I worked on a ‘Semantic Search’ product almost 10 years ago that used a neural network to do dimensional reduction and had inputs to the scoring function from the ‘gist vector’ and the residual word vector which was possible to calculate in that case because the gist vector was derived from the word vector and the transform was reversible.
I’ve seen papers in the literature which come to the same conclusion about what it takes to get good similarity results w/ older models as a significant amount of the meaning in text is in pointy words that might not be included in the gist vector, maybe you do better with an LLM since the vocabulary is huge.
Re: I used GPT to build a search tool for my second brain note-taking system
#3Would be nice to see some indication of how well it works in his case. I worked on a ‘Semantic Search’ product almost 10 years ago that used a neural network to do dimensional reduction and had inputs to the scoring function from the ‘gist vector’ and the residual word vector which was possible to calculate in that case because the gist vector was derived from the word vector and the transform was reversible. I’ve se…
I'd honestly argue that he might not have even needed OpenAI embeddings—any off-the-shelf Huggingface model would've sufficed.
Because of attention mechanisms, we no longer so heavily depend on the existence of those "pointy words," so generally, Transformers-based semantic search works quite well.
Re: I used GPT to build a search tool for my second brain note-taking system
#4Would be nice to see some indication of how well it works in his case. I worked on a ‘Semantic Search’ product almost 10 years ago that used a neural network to do dimensional reduction and had inputs to the scoring function from the ‘gist vector’ and the residual word vector which was possible to calculate in that case because the gist vector was derived from the word vector and the transform was reversible. I’ve se…
I'd honestly argue that he might not have even needed OpenAI embeddings—any off-the-shelf Huggingface model would've sufficed. Because of attention mechanisms, we no longer so heavily depend on the existence of those "pointy words," so generally, Transformers-based semantic search works quite well.
I was thinking RoBERTa 3, longformer or Big Bird would be a good choice for this, though having any limit on the attention window is a weakness.
Re: I used GPT to build a search tool for my second brain note-taking system
#5These "augmented intelligence" applications are so exciting to me. I'm not as interested in autonomous artificial intelligence. Computers are tools to make my life easier, not meant to lead their own lives!
There's a big up-front cost of building a notes database for this application, but it illustrates the point nicely: encode a bunch of data ("memories"), and use an AI like GPT to retrieve information ("remembering"). It's not a fundamentally different process from what we do already, but it replaces the need for me to spend time on an automatable task.
I'm excited to see what humans spend our time doing once we've offloaded the boring dirty work to AIs.
Re: I used GPT to build a search tool for my second brain note-taking system
#6Would be nice to see some indication of how well it works in his case. I worked on a ‘Semantic Search’ product almost 10 years ago that used a neural network to do dimensional reduction and had inputs to the scoring function from the ‘gist vector’ and the residual word vector which was possible to calculate in that case because the gist vector was derived from the word vector and the transform was reversible. I’ve se…
I'd honestly argue that he might not have even needed OpenAI embeddings—any off-the-shelf Huggingface model would've sufficed. Because of attention mechanisms, we no longer so heavily depend on the existence of those "pointy words," so generally, Transformers-based semantic search works quite well.
I actually tried this last year, before OpenAI released their cheaper embeddings v2 in december. From my experiments, when compared to Bert embeddings (or recent variation of the model) the OpenAI embeddings are miles ahead when doing similarity search.
Re: I used GPT to build a search tool for my second brain note-taking system
#7I wonder whether your individually trained chat bot will be allowed to assert the Fifth Amendment right against self-incrimination to stop it from talking when the police interview it. And if it's allowed, do you or it decide whether whether to assert it? What if the two of you disagree?
Similar questions for civil trials, divorce proceedings, child custody....
Re: I used GPT to build a search tool for my second brain note-taking system
#8This is fascinating.
Can I train it on 5 years of stream of consciousness morning brain dumps and then say "write blah as me"?
Before I do that, I'd love to know if training data becomes part of the global knowledge base available to everyone..
Re: I used GPT to build a search tool for my second brain note-taking system
#9This is fascinating. Can I train it on 5 years of stream of consciousness morning brain dumps and then say "write blah as me"? Before I do that, I'd love to know if training data becomes part of the global knowledge base available to everyone..
These privacy considerations are highest-priority for any extended roll-out of LLM-based products.
Privacy on the side of model servers would be good. Open source models that can be run locally would be better.
Re: I used GPT to build a search tool for my second brain note-taking system
#10I wonder whether your individually trained chat bot will be allowed to assert the Fifth Amendment right against self-incrimination to stop it from talking when the police interview it. And if it's allowed, do you or it decide whether whether to assert it? What if the two of you disagree? Similar questions for civil trials, divorce proceedings, child custody....
Imagine a model that decides on its own to assert the Fifth on your behalf. But now imagine that AI decides to lock you out of your own system...