Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Re-implementing LangChain in 100 lines of code

blog.scottlogic.com

Post reply on HN
Print view View original

51–60 of 85 posts

Re: Re-implementing LangChain in 100 lines of code

#51

Post by crazyedgar »

nestorD wrote: post #50

For me Langchain is glue code between a lot of commonly used LLM building blocks and prompts. It is great to get a prototype 80% of the way there fast in order to validate an idea or run something short lived. I suspect that, if you want to go further (simpler code, better control message length, reliability, etc), you will be better served by implementing the functionality you need yourself.

There's no suspicion. Our experience shows that for the remaining 20% of work, LangChain actually gets in the way and must eventually be removed.

Re: Re-implementing LangChain in 100 lines of code

#52

Post by ryanjshaw »

rcme wrote: post #16

LangChain has been so frequently discussed that I thought it must be this amazing piece of software. I was recently reading about vector databases and how they can be used to provide context to LLMs. I came across a LangChain class called RetrievalQA, which takes in a vector database and a question and produces and answer based on documents stored in the vector db. My curiosity was piqued! How did it work? Well... it…

I had the same experience and kept wondering if I was missing something important. I'm not a fan of Python, so I was anxious about not using the thing everybody recommended, but for my project I ultimately went with what I know well (C#). I've happily had zero issues.

LangChain docs and tutorials were useful for understanding the popular practices for approaching AI-driven development, but the biggest challenge by far has been getting a baseline prompt and measuring performance of alternative implementations against that in a sensible way that doesn't break the bank. Mitchell Hashimotos's Prompt Engineering article [1] was way more helpful in this regard than anything I saw in LangChain.

To that end I've also been working on a tool to save me money by caching requests and responses, blocking unexpectedly expensive requests, keeping a granular history of requests for prompt cost analysis, etc. Maybe I should open source it and get some VC bux too?

[1] https://mitchellh.com/writing/prompt-engineering-vs-blind-pr...

Re: Re-implementing LangChain in 100 lines of code

#53

Post by vidarh »

gumby wrote: post #8
loveparade wrote: post #6

Am I the only one who is not convinced by the value proposition of langchain? 99% of it are interface definitions and implementations for external tools, most of which are super straightforward. I can write integrations for what my app needs in less than an hour myself, why bring in a heavily opinionated external framework? It kind of feels like the npm "left-pad" to me. Everyone just uses it because it seems popular…

> I can write integrations for what my app needs in less than an hour myself, Or just ask ChatGPT to do it... Joking aside, I think 'npm "left-pad"' describes it perfectly.

Most of the code my OpenAI API experiments run on was written by ChatGPT.

Re: Re-implementing LangChain in 100 lines of code

#54

Post by danvass »

fbrncci wrote: post #14

I work with Langchain on a daily basis now, and so often I find myself asking; do I really need a whole LLM framework for this? At this point, the assistant I am writing, will likely be more stable rewritten in pure Python. The deeper and more complex the application becomes, the more of a risk Langchain seems to become to keeping it maintainable. But even at less complex levels, if I want to do this: 1. Have a huge…

We've been building LLM chains for over a year and found that whilst for simple use-cases it's easy enough to grab a couple of APIs. However, having a Notebook experience built for iterating and collaborating whilst managing the complexity is something we've seen companies care deeply about.

Re: Re-implementing LangChain in 100 lines of code

#55

Post by chaxor »

theturtletalks wrote: post #15
fbrncci wrote: post #14

I work with Langchain on a daily basis now, and so often I find myself asking; do I really need a whole LLM framework for this? At this point, the assistant I am writing, will likely be more stable rewritten in pure Python. The deeper and more complex the application becomes, the more of a risk Langchain seems to become to keeping it maintainable. But even at less complex levels, if I want to do this: 1. Have a huge…

Won’t ChatGPT eventually eat your lunch? Once ChatGPT allows uploading documents (embeddings), what good will your app be? The tools you’re talking about like math, wiki, or search are already built as plugins on ChatGPT. I see so many AI apps being built, but I think ChatGPT will be general enough to cover 85-90% of use-cases using the chat UI.

The most important aspect of langchain is NOT using OpenAI for the LM.

The most useful aspect of using langchain is to use it with Galpaca (or vicuna/koala/etc) to spin up an assistant for your home.

This way, you can push all of your files through it - even petabytes or terabytes of files, at a fraction of the cost - and have it organize things for you. No privacy problems, no extreme costs, just ease of use, low latency, offline, blazingly fast beauty. That's at least the trajectory.

Meta may soon release an improvement to Galactica similar to Galpaca (GeorgiaTech attempt) more officially (perhaps with more multimodal focus), which will likely improve upon the llama based models even further.

ChatGPT is just one model among many here, and it's not even the first to use RLHF (Deepmind, as usual, was a bit earlier).

The simple task of downloading Redpamajas/thePile/etc and getting a vector db for it locally, and enhancing it with local files effectively brings a local Google to everyone, and it may only require a decent spinning disk HD for the DB storage with the typical langchain LLM setup to have a completely local 'jarvis'-like assistant. (Sure, I know some people care about 'news'-like info that requires connectivity, but most things don't)

Re: Re-implementing LangChain in 100 lines of code

#57

Post by tholor »

iamflimflam1 wrote: post #47
ukuina wrote: post #28

I cannot praise Deepset Haystack enough for how simple they make things compared to LangChain, between the Preprocessor, the Reader/Retriever, and the PromptNode - the APIs, docs, and tutorials are quite easy to modify to your use-case. Not affiliated, just a happy defector from LangChain.

Is it open source? Wasn’t clear from the website.

Yes! https://github.com/deepset-ai/haystack

Re: Re-implementing LangChain in 100 lines of code

#58

Post by aantti »

iamflimflam1 wrote: post #47
ukuina wrote: post #28

I cannot praise Deepset Haystack enough for how simple they make things compared to LangChain, between the Preprocessor, the Reader/Retriever, and the PromptNode - the APIs, docs, and tutorials are quite easy to modify to your use-case. Not affiliated, just a happy defector from LangChain.

Is it open source? Wasn’t clear from the website.

Yes, Haystack is Apache 2.0 :) https://haystack.deepset.ai

Re: Re-implementing LangChain in 100 lines of code

#59

Post by havercosine »

Personal experience: was using LangChain and its output parsers for getting structured data. It was having a very high error rates (probably prompt was becoming too long and confusing). But it is a just prompt + some parsing logic. Replaced it with straight asking openAI GPT for json that matches some Rust struct / Python data-class. The errors went down and got one extra dependency out from the project. Tried to use its self hosted embeddings but the implementation (strangely) seemed tied to something called Run-house.

Not to belittle the library, but most of it is a very thin wrapper classes that reek of premature abstraction, couple with hit-n-miss docs. At this point, given the hype, it is primarily optimized for cooking up demoes quickly. But not sure if the valuations or production use is justified.

Re: Re-implementing LangChain in 100 lines of code

#60

Post by shri_krishna »

For the calculator tool I suggest instead to just generate Javascript as an output with temperature set to 0 (system prompt set to something along the lines of: "Generate native Javascript code only. Don't provide any explanations. Don't import any extraneous libraries") and then eval that Javascript code in a VM. Deno is a good candidate for this as it has good security settings with access to filesystem and network turned off by default. You can use something like deno-vm [1] to execute it separate from your running process too. Setting GPT-4 as model works even better. I have seen it perform better than Wolfram Alpha in many cases so I am wondering why OpenAI chose to integrate with Wolfram Alpha for this. GPT-4 was able to solve some really complex math problems I threw at it.

[1]: https://www.npmjs.com/package/deno-vm

Post reply on HN