Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we no longer use LangChain for building our AI agents

octomind.dev

Post reply on HN
Print view View original

181–190 of 307 posts

Re: Why we no longer use LangChain for building our AI agents

#181

Post by randomdata »

danielmarkbruce wrote: post #4

Yup. The problem with frameworks is they assume (historically mostly but not always correctly) that layers of abstraction mean one can forget about the layers below. This just doesn't work with LLMs. The systems are closer to biology or something.

It often took quite a long time for those historic frameworks to get the abstraction right. Survivorship bias sees us forget all the failed attempts.

I'm unconvinced there is no room for a framework here because LLMs are somehow special. LangChain just missed the mark. Unsurprisingly so, it being an early attempt, not to mention predating general availability of the LLM chatbots that have come to define the landscape.

Re: Why we no longer use LangChain for building our AI agents

#182

Post by authorfly »

wongarsu wrote: post #151
moffkalast wrote: post #134

Earlier quoted context omitted.

GPT-2 was not a fairly good chat model, it was a completely incoherent completion model. GPT-3 was not much better overall (take any entry level 1B sized model you can find today and it'll steamroll it in every way, hell probably even smaller ones), and the public at large never really had any access to it, I vaguely recall GPT 3 being locked behind an approval only paid API or something unfeasible like that. Nobody…

OpenAI had a real issue with making (for their time) great models but streching their rollout over months. They gave access to press and some twitter users, everyone else had to apply for their use case only to be put on the waitlist. That completely killed any momentum. The first version of ChatGPT wasn't a huge leap from simulating chat with instruction-tuned GPT 3.5, the real innovation was scaling it to the point…

The main reason ChatGPT took off was: 1) Response time of the API of that quality was 10x quicker than the Davinci-instruct-3 model that was released in summer 2022, making interaction more feasible with lower wait times and with concurrency 2) OpenAI strictly banned chat applications on the GPT API; even summarising with more than 150 tokens required your to submit a use case for review; I built an app around this in October 2022, got through the review, and it was then pointless as everybody could just use ChatGPT for the purposes of my apps new feature).

It was not possible for anybody to have just whacked the instruct models of GPT-3 into an interface for both the restrictions and latency issues that existed prior to ChatGPT. I agree with you on instruct vs ChatGPT and would further say the real innovation was entirely systematic, scaling and changing the interface. Instruct tuning was far more impactful than conversational model tuning because instruct enabled so many synthesizing use cases beyond the training data.

Re: Why we no longer use LangChain for building our AI agents

#183

Post by Calavar »

avereveard wrote: post #131
kgeist wrote: post #115

Earlier quoted context omitted.

>Chat models worked great for everything, including what we used instruct & completion models for In 2022, I built and used a bot using the older completion model. After GPT3.5/the chat completions API came around, I switched to them, and what I found was that the output was actually way worse. It started producing all those robotic "As an AI language model, I cannot..." and "It's important to note that..." all the t…

yeah gpt 3.5 just worked. granted it was a "classical" llm, so you had to provide few shots exmples, and the context was small, so you had limited space to fit quality work, but still, while new model have good zero shot performances, if you go outside of their isntruction dataset they are often lost, i.e. gpt4: "I've ten book and I read three, how many book I have?" "You have 7 books left to read. " and gpt4o: "shro…

I disagree about those questions being good examples of GPT4 pitfalls.

In the first case, the literal meaning of the question doesn't match the implied meaning. "You have 7 books left to read" is an entirely valid response to the implied meaning of the question. I could imagine a human giving the same response.

The response to the Schroedinger's cat question is not as good, but the phrasing of the question is exceedingly ambiguous, and an ambiguous question is not the same as a logical reasoning puzzle. Try asking this question to humans. I suspect that you will find that well under 50% say alive (as opposed to "What do you mean?" or some other attempt to disambiguate the question).

Re: Why we no longer use LangChain for building our AI agents

#184

Post by fforflo »

andix wrote: post #147
fforflo wrote: post #133

LLM frameworks like LangChain are causing a java-fication or Python . Do you want a banana? You should first create the universe and the jungle and use dependency injection to provide every tree one at a time, then create the monkey that will grab and eat the banana.

Langchain was my first real contact with Python development, and it felt worse than Enterprise Java. I didn't know that OOP is so prominent in Python libraries, it looks like many devs are just copying the mistakes from Enterprise Java/.NET projects.

Well it's not:D Sure there are 4-5 fundamental classes in python libs but they're just fundamental ones. They don't impose an OOP approach all the way.

What you're alluding to is people coming from Java to Python in 2010+ and having a use-classes-for-everything approach.

Re: Why we no longer use LangChain for building our AI agents

#185

Post by authorfly »

muzani wrote: post #116
isaacfung wrote: post #92

Earlier quoted context omitted.

I am not sure what you mean by "turn these one-shot APIs into Markov chains." To me, langchain was mostly marketed as a framework that makes RAG easy by providing integration with all kinds of data sources(vector db, pdf, sql db, web search, etc). Also older models(including initial chatgpt) had limited context lengths. Langchain helped you to manage the conversation memory by splitting it up and storing the pieces i…

Yup, I meant "Markov chain" as a way to say state. The idea was that it was extremely complex to control state. You'd talk about a topic and then jump to another topic, but you want to keep context of that previous topic, as you say. Was RAG popular on release? Google Trends indicates it started appearing around April 2023. To be honest, I'm trying to reverse engineer its popularity, and I think there are better solu…

I don't think this is a sensible use of Markov chain because that has historic connotations in NLP for text prediction models and would not include external resources in that.

RAG has been popular for years including in models like BERT and T5 which can also make use of contextual content (either in the prompt, or through biasing output logits which GPT also supports). You can see the earliest formal work that gained traction (mostly in 2021 and 2022 by citation count) here - http://proceedings.mlr.press/v119/guu20a/guu20a.pdf - though in my group, we already had something similar in 2019 too.

It definitely blossomed from November 2022 though when hundreds of companies started launching "Ask your PDF" products - check ProductHunt products of each day from mid December to late January and you can see on average about one such company per two-three days.

Re: Why we no longer use LangChain for building our AI agents

#186

Post by sc077y »

Damn I built a RAG agent during the past 3 months and a half for my internship. And literally everyone in my company was asking me why I wasn't using llangchain or llamaindex like I was a lunatic. Everyone else that built a rag in my company used llangchain, one even went into prod.

I kept telling them that it works well if you have a standard usage case but the second you need to something a little original you have to go through 5 layers of abstraction just to change a minute detail. Furthermore, you won't really understand every step in the process, so if any issue arises or you need to be improve the process you will start back at square 1.

This is honestly such a boost of confidence.

Re: Why we no longer use LangChain for building our AI agents

#187

Post by ramoz »

sc077y wrote: post #186

Damn I built a RAG agent during the past 3 months and a half for my internship. And literally everyone in my company was asking me why I wasn't using llangchain or llamaindex like I was a lunatic. Everyone else that built a rag in my company used llangchain, one even went into prod. I kept telling them that it works well if you have a standard usage case but the second you need to something a little original you have…

Wise perspective from an intern. The type of pragmatism we love.

Re: Why we no longer use LangChain for building our AI agents

#188

Post by Zambyte »

fforflo wrote: post #176
spywaregorilla wrote: post #143

Earlier quoted context omitted.

I feel like most of this complaint is about OOP, not java.

OOP is Java, and Java is OOP, right? My point is to follow a dogmatic OOP approach (think all the nouns like Agent, Prompt, etc.) to model something rather sequential.

Not if Alan Kay has anything to say about it.

Re: Why we no longer use LangChain for building our AI agents

#189

Post by diggan »

fforflo wrote: post #176
spywaregorilla wrote: post #143

Earlier quoted context omitted.

I feel like most of this complaint is about OOP, not java.

OOP is Java, and Java is OOP, right? My point is to follow a dogmatic OOP approach (think all the nouns like Agent, Prompt, etc.) to model something rather sequential.

No, you can do OOP without having to use Java, but you cannot really do Java without at least some OOP concepts.

I'm guessing only Smalltalk rivals Java in OOP-ness, as in Smalltalk literally everything is an object, while in Java only most things are objects.

Re: Why we no longer use LangChain for building our AI agents

#190

Post by refulgentis »

ssabev wrote: post #146
refulgentis wrote: post #37

Earlier quoted context omitted.

I have a consumer app that swaps between the 5 bigs and wholeheartedly agree, except, God help you if you're doing Gemini. I somewhat regret hacking it into the same concepts as everyone else. I should have built stronger separation boundaries with more general abstractions. It works fine, I haven't had any critical bugs / mistakes, but it's really nasty once you get to the actual JSON you'll send. Google's was 100%…

Would recommend just picking up a gateway that you can deploy and act as an OpenAI compatible endpoint. We built something like this for ourselves here -> https://www.npmjs.com/package/@kluai/gateway?activeTab=readm... . Documentation is a bit sparse but TL;DR - deploy it in a cloudflare worker and now you can access about 15 providers (the one that matter - OpenAI, Cohere, Azure, Bedrock, Gemini, etc) all with the s…

Wow; this is really nice work, I wish you deep success.
Post reply on HN