we're still in the stage of LLM adoption when we can have "eye-opening" simple discoveries weekly. Langchain has momentum because of this, as a library of simple ideas. this period will end, and if they don't figure out the next step they're gone
Re-implementing LangChain in 100 lines of code
31–40 of 85 posts
Re: Re-implementing LangChain in 100 lines of code
#32As someone who has created several LLM-based applications running in production, my personal experience with langchain has been that it is too high of an abstraction for steps that in the end are actually fairly simple. And as soon as you want to slightly modify something to better accomodate your use-case, you are trapped in layers & layers of Python boiler plate code and unnecessary abstractions. Maybe our llm appl…
Comparing Langchain to Hugging Face Transformers is apples and oranges. One is for research, one is for production. Production ML requires more abstraction, not less.
Re: Re-implementing LangChain in 100 lines of code
#33There’s always DSP for those who need a lightweight but powerful programming model — not a library of predefined prompts and integrations. It’s a very different experience from the hand-holding of LangChain, but it packs reusable magic in generic constructs like annotate, compile, etc that work with arbitrary programs. https://github.com/stanfordnlp/dsp/
Didn't know about this. Looks promising!
Re: Re-implementing LangChain in 100 lines of code
#34LangChain has been so frequently discussed that I thought it must be this amazing piece of software. I was recently reading about vector databases and how they can be used to provide context to LLMs. I came across a LangChain class called RetrievalQA, which takes in a vector database and a question and produces and answer based on documents stored in the vector db. My curiosity was piqued! How did it work? Well... it…
If it makes you feel any better, the researchers building these things don't really know what they're doing either. They just throw data and compute at the problem and hope for the best.
Re: Re-implementing LangChain in 100 lines of code
#35Earlier quoted context omitted.
It got a lot of traction on non-coders thinking it’s doing some magic, but if you read the source it boils down to some brittle prompts and lots of Python class boilerplate. There’s some useful parts though
Just out of curiosity, what are the useful parts?
The class structure provides a clean API for building on, even if the internals are basic. With some refinement, it could be a good starting point for more advanced models.
Re: Re-implementing LangChain in 100 lines of code
#36Am I the only one who is not convinced by the value proposition of langchain? 99% of it are interface definitions and implementations for external tools, most of which are super straightforward. I can write integrations for what my app needs in less than an hour myself, why bring in a heavily opinionated external framework? It kind of feels like the npm "left-pad" to me. Everyone just uses it because it seems popular…
For us LangChain actually caused more problems than it solved. We had a system in production which after working fine a few weeks suddenly started experiencing frequent failures (more than 30% of requests). On digging it seems that LangChain sets a default timeout of 60 seconds for every requests. And this behaviour isn't documented! Such spurious decisions made by LangChain are everywhere, and will all eventually come back to bite. In the end we replaced everything with vanilla request clients. Definitely not recommended to build a system on a library that provides very limited value while hiding a huge amount of details and decisions from you.
Re: Re-implementing LangChain in 100 lines of code
#37LangChain has been so frequently discussed that I thought it must be this amazing piece of software. I was recently reading about vector databases and how they can be used to provide context to LLMs. I came across a LangChain class called RetrievalQA, which takes in a vector database and a question and produces and answer based on documents stored in the vector db. My curiosity was piqued! How did it work? Well... it…
It basically gathers text that are similar to what you are asking and feed it into the prompt, yes. No magic. The worse part is that if you ask “please get me the summary to this doc” it will actually search the vector db using the entire question. It’s not very smart. Depending on how you split the embedding you could end up with a bunch of crap
Re: Re-implementing LangChain in 100 lines of code
#38I've been working with langchain and llamaindex and did notice that it's a pretty hefty abstraction on top of pretty simple concepts and I also eventually ended up dropping both and simply write the underlying code without the framework on top.
Ditto. But with faiss and openai api.
Re: Re-implementing LangChain in 100 lines of code
#39LangChain has been so frequently discussed that I thought it must be this amazing piece of software. I was recently reading about vector databases and how they can be used to provide context to LLMs. I came across a LangChain class called RetrievalQA, which takes in a vector database and a question and produces and answer based on documents stored in the vector db. My curiosity was piqued! How did it work? Well... it…
If it makes you feel any better, the researchers building these things don't really know what they're doing either. They just throw data and compute at the problem and hope for the best.
I wouldn't say that. If it was that simple, the credits of GPT-4 wouldn't be this long: https://openai.com/contributions/gpt-4
Re: Re-implementing LangChain in 100 lines of code
#40LangChain has been so frequently discussed that I thought it must be this amazing piece of software. I was recently reading about vector databases and how they can be used to provide context to LLMs. I came across a LangChain class called RetrievalQA, which takes in a vector database and a question and produces and answer based on documents stored in the vector db. My curiosity was piqued! How did it work? Well... it…
The default "document splitter" just splits along double newlines. It took me aback when I realized they made an entire class around this.