Can anyone use the ChatGPT Retrieval plugin yet? Or is it limited to whitelisted beta testers?
Still whitelisted atm.
Retrieval in LangChain
31–40 of 71 posts
Re: Retrieval in LangChain
#32Earlier quoted context omitted.
If you put GPT-4 on a loop with access to the shell it manages to do whatever is needed to finish the job https://raw.githubusercontent.com/jla/gpt-shell/assets/examp...
My experience with GPT-4 has been really disappointing. It didn't feel like a step up from 3.5. As an example, I've been trying to use it to learn Zig since the official docs are ... spartan. And I've said, "here's my code, here's the error, what's wrong with it?" and it will go completely off the rails suggesting fixes that don't do anything (or are themselves wrong). In my case, understanding/fixing the code would…
If you are using GPT-4 to try to deal with the fact that technical documentation on the public internet is sparse for your topic of interest, you are likely to be disappointed, since GPT-4’s training set likely has the same problem, so you are, in effect, hoping it will fill in gaps in missing data, prompting hallucinations.
It’ll be much better on subjects where there is too much information on the public internet for a person to efficiently manage and sift through.
Re: Retrieval in LangChain
#33I've been playing around with sentence embeddings to search documents, but I wonder how useful they are as a natural language interface for a database. The way one might phrase a question might be very different content wise from how the document describes the answer. Maybe it might be possible to do some type of transform where the question is transformed into a possible answer and then turned into a embedding but I…
In your first paragraph, you are describing Hypothetical Document Embeddings (HyDE) [0]. I've tested it out, and in certain cases, it works amazingly well to get more complete answers.
[0] https://python.langchain.com/en/latest/modules/chains/index_...
Re: Retrieval in LangChain
#34Earlier quoted context omitted.
My experience with GPT-4 has been really disappointing. It didn't feel like a step up from 3.5. As an example, I've been trying to use it to learn Zig since the official docs are ... spartan. And I've said, "here's my code, here's the error, what's wrong with it?" and it will go completely off the rails suggesting fixes that don't do anything (or are themselves wrong). In my case, understanding/fixing the code would…
If you are using GPT-4 to try to deal with the fact that technical documentation on the public internet is sparse for your topic of interest, you are likely to be disappointed, since GPT-4’s training set likely has the same problem, so you are, in effect, hoping it will fill in gaps in missing data, prompting hallucinations. It’ll be much better on subjects where there is too much information on the public internet f…
I think you're right. My hope was that it could reason through the problem using knowledge from related sources like C and an understanding below the syntax of what was actually happening.
But it most certainly did not.
Re: Retrieval in LangChain
#35How can I use this with llama ?
Re: Retrieval in LangChain
#36Earlier quoted context omitted.
If you put GPT-4 on a loop with access to the shell it manages to do whatever is needed to finish the job https://raw.githubusercontent.com/jla/gpt-shell/assets/examp...
Yeah I can't wait to get API access to gpt-4, it is a stepwise more capable based on the stuff I've done with chatgpt on gpt-4. That said, even gpt-3.5 will try multiple routes to get to the same endpoint. It seems to get distracted pretty easily though.
> It seems to get distracted pretty easily though.
That’s true, gpt-4 is way more easy to guide with the system messages and it doesn’t forget the instructions as the conversation goes on.
Re: Retrieval in LangChain
#37Making retrieval really really good is part of the mission of LlamaIndex! Given a natural language input, find the best way to return a set of documents that is relevant to your LLM use case (question-answering, summarization, more complex queries too).
- We integrate with vector db's + ChatGPT Retrieval Plugin
- Submitted a Retrieval PR to langchain here: https://github.com/hwchase17/langchain/pull/2014
- would love to explore further integrations as a plugin in any outer agent system
Re: Retrieval in LangChain
#38Earlier quoted context omitted.
If you put GPT-4 on a loop with access to the shell it manages to do whatever is needed to finish the job https://raw.githubusercontent.com/jla/gpt-shell/assets/examp...
My experience with GPT-4 has been really disappointing. It didn't feel like a step up from 3.5. As an example, I've been trying to use it to learn Zig since the official docs are ... spartan. And I've said, "here's my code, here's the error, what's wrong with it?" and it will go completely off the rails suggesting fixes that don't do anything (or are themselves wrong). In my case, understanding/fixing the code would…
1. GPT4 is learning from the same spartan docs as you, likely
2. GPT4's training data likely doesn't include significant Zig use, since large parts of its training data cut off a few years ago. I use Rust and it doesn't know about any recently added Rust features, either.
This has interesting implications because it means people will gravitate towards languages/frameworks/libraries that GPT knows well, which means even less training data will be generated for the new stuff. This is a form of value lock-in.
Re: Retrieval in LangChain
#39Re: Retrieval in LangChain
#40Earlier quoted context omitted.
My experience with GPT-4 has been really disappointing. It didn't feel like a step up from 3.5. As an example, I've been trying to use it to learn Zig since the official docs are ... spartan. And I've said, "here's my code, here's the error, what's wrong with it?" and it will go completely off the rails suggesting fixes that don't do anything (or are themselves wrong). In my case, understanding/fixing the code would…
1. GPT4 is learning from the same spartan docs as you, likely 2. GPT4's training data likely doesn't include significant Zig use, since large parts of its training data cut off a few years ago. I use Rust and it doesn't know about any recently added Rust features, either. This has interesting implications because it means people will gravitate towards languages/frameworks/libraries that GPT knows well, which means ev…
> This has interesting implications because it means people will gravitate towards languages/frameworks/libraries that GPT knows well, which means even less training data will be generated for the new stuff. This is a form of value lock-in.
That's the kind of problem that most people are just failing to see. The usage of this models might not in itself be problematic, but the changes that it bring are often unexpected and too deep for us to see clearly now. And yet, people are rushing towards them at full speed.