Re-implementing LangChain in 100 lines of code • Hacker News Board

Re: Re-implementing LangChain in 100 lines of code

#21

Post by fzliu » Fri, May 05, 2023, 2:06 AM UTC

rcme wrote: post #16

LangChain has been so frequently discussed that I thought it must be this amazing piece of software. I was recently reading about vector databases and how they can be used to provide context to LLMs. I came across a LangChain class called RetrievalQA, which takes in a vector database and a question and produces and answer based on documents stored in the vector db. My curiosity was piqued! How did it work? Well... it…

LangChain is meant to reduce/remove the amount of boilerplate code needed to build a lot of applications with LLMs. I see your point, but I still think LangChain is useful for a particular segment of early stage developers.

Re: Re-implementing LangChain in 100 lines of code

#22

Post by theturtletalks » Fri, May 05, 2023, 2:12 AM UTC

fbrncci wrote: post #17

theturtletalks wrote: post #15

Earlier quoted context omitted.

Won’t ChatGPT eventually eat your lunch? Once ChatGPT allows uploading documents (embeddings), what good will your app be? The tools you’re talking about like math, wiki, or search are already built as plugins on ChatGPT. I see so many AI apps being built, but I think ChatGPT will be general enough to cover 85-90% of use-cases using the chat UI.

It's more or less an advanced personal project to stay on top of the LLM learning curve, rather than just being exposed to news and press releases. I also have an appetite for further wrapping my mind around all of this. I already work in the AI space as web-developer on the B2B & enterprise side of things. My opinion here is that there are going to be loads of use-cases and necessary plugins, which for privacy, lega…

I completely agree with you on AI being extensively integrated into existing apps and being leveraged that way.

But most of the AI apps I see are just like a “skin” on ChatGPT API.

I do think there is value in a universal chat UI that can connect to GPT-3 and other models.

Re: Re-implementing LangChain in 100 lines of code

#23

Post by fbrncci » Fri, May 05, 2023, 2:40 AM UTC

theturtletalks wrote: post #22

fbrncci wrote: post #17

Earlier quoted context omitted.

It's more or less an advanced personal project to stay on top of the LLM learning curve, rather than just being exposed to news and press releases. I also have an appetite for further wrapping my mind around all of this. I already work in the AI space as web-developer on the B2B & enterprise side of things. My opinion here is that there are going to be loads of use-cases and necessary plugins, which for privacy, lega…

I completely agree with you on AI being extensively integrated into existing apps and being leveraged that way. But most of the AI apps I see are just like a “skin” on ChatGPT API. I do think there is value in a universal chat UI that can connect to GPT-3 and other models.

I think that is a phase, which will come to pass, eventually most of the plugins and apps which are going to be popular, will likely run their own models or use open source models. Because the API calls for complex applications to OpenAI are currently far from economical. In the end you will have to charge the user, and that is going to be the crux. As a sole developer, doing loads of experiments on less than 50 documents, I am already crossing 50$ in API calls within a month. I can already run llama.cpp, but its just not good enough; but the cost would effectively be 0$ (not counting my hardware).

Re: Re-implementing LangChain in 100 lines of code

#24

Post by nbardy » Fri, May 05, 2023, 2:43 AM UTC

fbrncci wrote: post #14

I work with Langchain on a daily basis now, and so often I find myself asking; do I really need a whole LLM framework for this? At this point, the assistant I am writing, will likely be more stable rewritten in pure Python. The deeper and more complex the application becomes, the more of a risk Langchain seems to become to keeping it maintainable. But even at less complex levels, if I want to do this: 1. Have a huge…

I’m not convinced yet, but if they can provide nice templating and reusability.

There is plenty room for code reuse in prompting.

Re: Re-implementing LangChain in 100 lines of code

#25

Post by nextworddev » Fri, May 05, 2023, 2:44 AM UTC

rcme wrote: post #20

nextworddev wrote: post #18

Earlier quoted context omitted.

It got a lot of traction on non-coders thinking it’s doing some magic, but if you read the source it boils down to some brittle prompts and lots of Python class boilerplate. There’s some useful parts though

Just out of curiosity, what are the useful parts?

It’s good for building vector indices without worrying about writing adapters to milvus, pinecone, qdrant etc separately - in case you want to switch out later

Re: Re-implementing LangChain in 100 lines of code

#26

Post by KevinBenSmith » Fri, May 05, 2023, 2:44 AM UTC

lxe wrote: post #10

I've been working with langchain and llamaindex and did notice that it's a pretty hefty abstraction on top of pretty simple concepts and I also eventually ended up dropping both and simply write the underlying code without the framework on top.

Pretty much sums up my experience as well.

Re: Re-implementing LangChain in 100 lines of code

#27

Post by ukuina » Fri, May 05, 2023, 2:47 AM UTC

throwaway888abc wrote: post #3

justanotheratom wrote: post #2

Given that the company has $200 million valuation, that is $2 million per line of code! just kidding. Still, I would like to understand $200 million valuation of langchain.ai.

It's a great momentum to dominate eco system. Heard about Docker (that was more than http fetch wrapper) ?

I think we can safely say this did not end well for Docker.

Re: Re-implementing LangChain in 100 lines of code

#28

Post by ukuina » Fri, May 05, 2023, 2:50 AM UTC

I cannot praise Deepset Haystack enough for how simple they make things compared to LangChain, between the Preprocessor, the Reader/Retriever, and the PromptNode - the APIs, docs, and tutorials are quite easy to modify to your use-case.

Not affiliated, just a happy defector from LangChain.

Re: Re-implementing LangChain in 100 lines of code

#29

Post by fbrncci » Fri, May 05, 2023, 2:57 AM UTC

rcme wrote: post #16

LangChain has been so frequently discussed that I thought it must be this amazing piece of software. I was recently reading about vector databases and how they can be used to provide context to LLMs. I came across a LangChain class called RetrievalQA, which takes in a vector database and a question and produces and answer based on documents stored in the vector db. My curiosity was piqued! How did it work? Well... it…

But wait, there is more! In Langchain you can build constitutional chains ontop of your chains, to validate if the answer was really helpful, by doing just one more API call, with a new prompt, asking if the answer answered the question based on the initial prompt, in an helpful way! And if it didn't, revise the answer with another API call to be more helpful! And then you can chain these chains with even further API calls until you went through as much prompts you think are necessary to answer a single sentence question (What is the weather today on the moon?).

Re: Re-implementing LangChain in 100 lines of code

#30

Post by KevinBenSmith » Fri, May 05, 2023, 3:08 AM UTC

As someone who has created several LLM-based applications running in production, my personal experience with langchain has been that it is too high of an abstraction for steps that in the end are actually fairly simple.

And as soon as you want to slightly modify something to better accomodate your use-case, you are trapped in layers & layers of Python boiler plate code and unnecessary abstractions.

Maybe our llm applications haven’t been complex enough to warrent the use of langchain, but if that’s the case, then I wonder how many of such complex applications actually exist today.

-> Anyways, I came away feeling quite let down by the hype.

For my own personal workflow, a more “hackable” architecture would be much more valuable. Totally fine if that means it’s less “general”. As a comparison, I remember the early days of HugginfaceTransformers where they did not try to create a 100% high-level general abstraction on top of every conceivable Neural Network architecture. Instead, each model architecture was somewhat separate from one another, making it much easier to “hack” it.