Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Retrieval in LangChain

blog.langchain.dev

Post reply on HN
Print view View original

11–20 of 71 posts

Re: Retrieval in LangChain

#11

Post by visarga »

KrugerDunnings wrote: post #5

I've been playing around with sentence embeddings to search documents, but I wonder how useful they are as a natural language interface for a database. The way one might phrase a question might be very different content wise from how the document describes the answer. Maybe it might be possible to do some type of transform where the question is transformed into a possible answer and then turned into a embedding but I…

> The way one might phrase a question might be very different content wise from how the document describes the answer.

If the embeddings are worth their salt, then they should not be influenced by paraphrasing with different words. Try the OpenAI embeddings or sbert.net embedding models.

Re: Retrieval in LangChain

#12

Post by mark_l_watson »

Spivak wrote: post #4

Really excited to see LangChain moving really fast in this space. They turn your favorite Llm into a real boy that can do real work. Some "agents" in their vernacular that I've built. * A reminder system that can take a completely free-form English description and turn it into a precise date and time and schedule it with an external scheduler. * A tool that can take math either in English or ascii like y = 2x^2 + ln(…

I share your enthusiasm for LangChain (as well as LlamaIndex). I don’t remember being as obsessed by any new technology. I am writing a book on the subject [1] but to be honest the documentation and available examples are so very good, that my book has turned into just a write up of my own little projects.

I agree that some things really are trivial to implement and I think this opens the door to non-programmers who know a little Python or JavaScipt to scratch their own itches and build highly personalized systems.

[1] https://leanpub.com/langchain

Re: Retrieval in LangChain

#13

Post by pumanoir »

Spivak wrote: post #4

Really excited to see LangChain moving really fast in this space. They turn your favorite Llm into a real boy that can do real work. Some "agents" in their vernacular that I've built. * A reminder system that can take a completely free-form English description and turn it into a precise date and time and schedule it with an external scheduler. * A tool that can take math either in English or ascii like y = 2x^2 + ln(…

These examples are great, but the chess engine sounds specially interesting and can't think of how I'd do it with langchain. Do you have a git link or something written down, on how you accomplished this?

Re: Retrieval in LangChain

#15

Post by jncraton »

KrugerDunnings wrote: post #5

I've been playing around with sentence embeddings to search documents, but I wonder how useful they are as a natural language interface for a database. The way one might phrase a question might be very different content wise from how the document describes the answer. Maybe it might be possible to do some type of transform where the question is transformed into a possible answer and then turned into a embedding but I…

Embeddings can be trained specifically to cause questions and content including their answers to have similar representations in latent space. This has been used this to create QA retrieval systems. Here's one commonly used example:

https://huggingface.co/sentence-transformers/multi-qa-MiniLM...

Re: Retrieval in LangChain

#16

Post by jcims »

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior.

The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take.

This is just scratching the surface. I've seen it do some crazy stuff with the AWS CLI. And this is just with GPT-3.5, I don't have access to GPT-4 yet and it clearly has better capabilities.

https://pastebin.com/qJsbufVj

Re: Retrieval in LangChain

#18

Post by robbintt »

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

What is the aws cli stuff? I was thinking about writing a terraform agent tool.

Re: Retrieval in LangChain

#20

Post by antibasilisk »

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

That pastebin is mindboggling.
Post reply on HN