Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used GPT to build a search tool for my second brain note-taking system

reasonabledeviations.com

Post reply on HN
Print view View original

11–20 of 103 posts

Re: I used GPT to build a search tool for my second brain note-taking system

#11

Post by feanaro »

michaericalribo wrote: post #9
tra3 wrote: post #8

This is fascinating. Can I train it on 5 years of stream of consciousness morning brain dumps and then say "write blah as me"? Before I do that, I'd love to know if training data becomes part of the global knowledge base available to everyone..

These privacy considerations are highest-priority for any extended roll-out of LLM-based products. Privacy on the side of model servers would be good. Open source models that can be run locally would be better.

I personally think anything server-side is unacceptable. Only open source and local will fly.

Re: I used GPT to build a search tool for my second brain note-taking system

#12

Post by ilaksh »

tra3 wrote: post #8

This is fascinating. Can I train it on 5 years of stream of consciousness morning brain dumps and then say "write blah as me"? Before I do that, I'd love to know if training data becomes part of the global knowledge base available to everyone..

This is not a fine-tuning example. It's an embedding search example. You use the embeddings to search for relevant knowledgebase chunks and then include them in the prompt. Which goes to the original model, not a model that you have trained more.

This is popular because it's much much easier to do effectively than fine tuning and the OpenAI model is very capable of integrating kb snippets into a response. What I have heard is that it's easy to overdo fine tuning with OpenAI's model and makes more sense when you want a different format of response rather than just pulling in some content.

Having said all of that, they do have a fine-tuning endpoint and I am guessing if you find the right parameters and give it a lot of properly formatted training data then it will be able to do an okay job. I have the impression it is not easy to do either of those things quite right though.

As far as privacy, no they will not share your data when you use the API. ChatGPT is different, they ARE using the inputs to train the model.

Re: I used GPT to build a search tool for my second brain note-taking system

#13

Post by PartiallyTyped »

tra3 wrote: post #8

This is fascinating. Can I train it on 5 years of stream of consciousness morning brain dumps and then say "write blah as me"? Before I do that, I'd love to know if training data becomes part of the global knowledge base available to everyone..

I have considered training a model on about a year’s conversations from my little community’s discord server and ask it so synthesise sentences as if I was writing them.

Re: I used GPT to build a search tool for my second brain note-taking system

#14

Post by leobg »

Slight overkill to use GPT, though it works for the author and I can see that it’s the low hanging fruit, being available as an API. But this can also be done locally, using SBERT, or even (faster, though less powerful) fastText.

Also, it’s helpful not to cut paragraphs into separate pieces, but rather to use a sliding window approach, where each paragraph retains the context of what came before, and/or the breadcrumbs of its parent headlines.

Re: I used GPT to build a search tool for my second brain note-taking system

#15

Post by DavidPiper »

michaericalribo wrote: post #10
sowbug wrote: post #7

I wonder whether your individually trained chat bot will be allowed to assert the Fifth Amendment right against self-incrimination to stop it from talking when the police interview it. And if it's allowed, do you or it decide whether whether to assert it? What if the two of you disagree? Similar questions for civil trials, divorce proceedings, child custody....

Imagine a model that decides on its own to assert the Fifth on your behalf. But now imagine that AI decides to lock you out of your own system...

We already have those. They're called Google, Microsoft and Apple.

Re: I used GPT to build a search tool for my second brain note-taking system

#16

Post by crosen99 »

ilaksh wrote: post #12
tra3 wrote: post #8

This is fascinating. Can I train it on 5 years of stream of consciousness morning brain dumps and then say "write blah as me"? Before I do that, I'd love to know if training data becomes part of the global knowledge base available to everyone..

This is not a fine-tuning example. It's an embedding search example. You use the embeddings to search for relevant knowledgebase chunks and then include them in the prompt. Which goes to the original model, not a model that you have trained more. This is popular because it's much much easier to do effectively than fine tuning and the OpenAI model is very capable of integrating kb snippets into a response. What I have…

> Having said all of that, they do have a fine-tuning endpoint and I am guessing if you find the right parameters and give it a lot of properly formatted training data then it will be able to do an okay job.

Unfortunately, the fine-tuning API cannot be used to add knowledge to the model. It only helps condition the model to a certain response pattern using the knowledge it already has.

Re: I used GPT to build a search tool for my second brain note-taking system

#17

Post by qwertox »

sowbug wrote: post #7

I wonder whether your individually trained chat bot will be allowed to assert the Fifth Amendment right against self-incrimination to stop it from talking when the police interview it. And if it's allowed, do you or it decide whether whether to assert it? What if the two of you disagree? Similar questions for civil trials, divorce proceedings, child custody....

This is a topic which really deserves a lot more attention, as in: from magazines to newspapers to talk shows. Seems like an appropriate time to get it on the agenda before governments opt to decide on their own.

Re: I used GPT to build a search tool for my second brain note-taking system

#18

Post by roywiggins »

sowbug wrote: post #7

I wonder whether your individually trained chat bot will be allowed to assert the Fifth Amendment right against self-incrimination to stop it from talking when the police interview it. And if it's allowed, do you or it decide whether whether to assert it? What if the two of you disagree? Similar questions for civil trials, divorce proceedings, child custody....

How could it have a right not to self-incriminate when it can't be tried for a crime? An AI can't be indicted or convicted.

Humans can be required to testify too if they're immunized.

Re: I used GPT to build a search tool for my second brain note-taking system

#19

Post by sokoloff »

roywiggins wrote: post #18
sowbug wrote: post #7

I wonder whether your individually trained chat bot will be allowed to assert the Fifth Amendment right against self-incrimination to stop it from talking when the police interview it. And if it's allowed, do you or it decide whether whether to assert it? What if the two of you disagree? Similar questions for civil trials, divorce proceedings, child custody....

How could it have a right not to self-incriminate when it can't be tried for a crime? An AI can't be indicted or convicted. Humans can be required to testify too if they're immunized.

With limitations. I cannot be compelled to testify against my wife (and probably not against my kids, though I’m unsure of that [edit: that seems to vary by state currently]), even if I personally am granted immunity.

Re: I used GPT to build a search tool for my second brain note-taking system

#20

Post by bbor »

sowbug wrote: post #7

I wonder whether your individually trained chat bot will be allowed to assert the Fifth Amendment right against self-incrimination to stop it from talking when the police interview it. And if it's allowed, do you or it decide whether whether to assert it? What if the two of you disagree? Similar questions for civil trials, divorce proceedings, child custody....

Wait is this a serious concern or a joke? I think asking a chat bot whether it’s admissible is the same as asking someone’s handwritten diary if it’s admissible.
Post reply on HN