Retrieval in LangChain • Hacker News Board

Re: Retrieval in LangChain

#21

Post by amitport » Mon, Mar 27, 2023, 3:38 PM UTC

KrugerDunnings wrote: post #5

I've been playing around with sentence embeddings to search documents, but I wonder how useful they are as a natural language interface for a database. The way one might phrase a question might be very different content wise from how the document describes the answer. Maybe it might be possible to do some type of transform where the question is transformed into a possible answer and then turned into a embedding but I…

"The way one might phrase a question might be very different content wise from how the document describes the answer."

You have late-interaction models, which replace the dot product with a few transformer layers and are able to learn complex semantics.

Of course this would adversely affect latency and embedding size, so you might want to compress and cache the answers, hence (shameless plug):

https://aclanthology.org/2022.acl-long.457/

Re: Retrieval in LangChain

#22

Post by badloginagain » Mon, Mar 27, 2023, 3:47 PM UTC

antibasilisk wrote: post #20

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

That pastebin is mindboggling.

I'm glad our AI overlords are as bamboozled by python vs. python3 as us lowly humans

Re: Retrieval in LangChain

#23

Post by px43 » Mon, Mar 27, 2023, 4:00 PM UTC

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

Can you share the code for llm.py?

Re: Retrieval in LangChain

#24

Post by jcims » Mon, Mar 27, 2023, 4:06 PM UTC

px43 wrote: post #23

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

Can you share the code for llm.py?

    #!/home/ubuntu/venv/bin/python3.10
    from langchain.agents import load_tools
    from langchain.agents import initialize_agent
    from langchain.chat_models import ChatOpenAI
     
     
    llm = ChatOpenAI(model='gpt-3.5-turbo',temperature=0)
    tools = load_tools(['python_repl', 'requests', 'terminal', 'wolfram-alpha', 'serpapi', 'wikipedia', 'human',  'pal-math', 'pal-colored-objects'], llm=llm)
     
    agent = initialize_agent(tools, llm, agent="chat-zero-shot-react-description", verbose=True)
     
    agent.run("Ask the human what they want to do")

Note that you'll need to get api keys for openai and serpapi and a app id from wolfram-alpha.

Re: Retrieval in LangChain

#25

Post by jcims » Mon, Mar 27, 2023, 4:10 PM UTC

robbintt wrote: post #18

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

What is the aws cli stuff? I was thinking about writing a terraform agent tool.

You install and configure the CLI to run locally, then ask it to do something. For example, ask it to create a website using s3 static website feature. It creates the bucket, creates the content, uploads the content, configures the bucket static website features and configures the permissions on the bucket and content.

I just started tinkering with terraform, which it seems to understand fairly well.

Re: Retrieval in LangChain

#26

Post by fudged71 » Mon, Mar 27, 2023, 4:12 PM UTC

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

About how long did this take to run?

It would be great if it also summarized what the error was, what was the fix, and how to run the code that it created. That’s all in the output but could be pulled out at the end.

Re: Retrieval in LangChain

#27

Post by ArenaSource » Mon, Mar 27, 2023, 4:13 PM UTC

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

If you put GPT-4 on a loop with access to the shell it manages to do whatever is needed to finish the job

https://raw.githubusercontent.com/jla/gpt-shell/assets/examp...

Re: Retrieval in LangChain

#28

Post by jcims » Mon, Mar 27, 2023, 4:20 PM UTC

fudged71 wrote: post #26

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

About how long did this take to run? It would be great if it also summarized what the error was, what was the fix, and how to run the code that it created. That’s all in the output but could be pulled out at the end.

A minute or two.

You can ask it to summarize things if you like. It sometimes forgets to do so, however.

Re: Retrieval in LangChain

#29

Post by jcims » Mon, Mar 27, 2023, 4:21 PM UTC

ArenaSource wrote: post #27

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

If you put GPT-4 on a loop with access to the shell it manages to do whatever is needed to finish the job https://raw.githubusercontent.com/jla/gpt-shell/assets/examp...

Yeah I can't wait to get API access to gpt-4, it is a stepwise more capable based on the stuff I've done with chatgpt on gpt-4.

That said, even gpt-3.5 will try multiple routes to get to the same endpoint. It seems to get distracted pretty easily though.

Re: Retrieval in LangChain

#30

Post by hellcow » Mon, Mar 27, 2023, 4:22 PM UTC

ArenaSource wrote: post #27

jcims wrote: post #16

Enabling the 'terminal' and 'python-repl' tools in a langchain agent demonstrates some pretty remarkable behavior. The link below is the transcript of a session in which I asked the agent to create a hello world script and executes it. The only input I provide is on line 17. Everything else is the langchain agent iteratively taking an action, observing the results and deciding the next action to take. This is just sc…

If you put GPT-4 on a loop with access to the shell it manages to do whatever is needed to finish the job https://raw.githubusercontent.com/jla/gpt-shell/assets/examp...

My experience with GPT-4 has been really disappointing. It didn't feel like a step up from 3.5.

As an example, I've been trying to use it to learn Zig since the official docs are ... spartan. And I've said, "here's my code, here's the error, what's wrong with it?" and it will go completely off the rails suggesting fixes that don't do anything (or are themselves wrong).

In my case, understanding/fixing the code would have required GPT-4 to know the difference between allocating on the stack/heap and the lifetimes of pointers. It never even approached the right solution.

I haven't yet gotten it to help me in even a single instance. Every suggestion is wrong or won't compile, and it can't reason through the errors iteratively to find a fix. I'm sure this has to do with a small sample of Zig code in its training set, but I reckon an expert C coder could have spotted the bug instantly.