Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we no longer use LangChain for building our AI agents

octomind.dev

Post reply on HN
Print view View original

161–170 of 307 posts

Re: Why we no longer use LangChain for building our AI agents

#161

Post by pietro72ohboy »

shpx wrote: post #135
pietro72ohboy wrote: post #104

Earlier quoted context omitted.

Chat models were not invented with ChatGPT. Conversational search and AI was a well-established field of study well before ChatGPT. It is remarkable how many people unfamiliar with the field think ChatGPT was the first chat model. It may be the first widely-popular chat model but it certainly isn’t the first

People call the first actually useful thing the first thing, that's not surprising or wrong.

That statement is patently incorrect. While the 'usefulness' of something can be subjective, the date of creation is an absolute, immutable fact.

Re: Why we no longer use LangChain for building our AI agents

#162

Post by turbocon »

fforflo wrote: post #133

LLM frameworks like LangChain are causing a java-fication or Python . Do you want a banana? You should first create the universe and the jungle and use dependency injection to provide every tree one at a time, then create the monkey that will grab and eat the banana.

Id just like to point out the source of the Gorilla Banana problem is Joe Armstrong. He really had an amazing way of explain complex problems in a simple way.

https://www.johndcook.com/blog/2011/07/19/you-wanted-banana/

Re: Why we no longer use LangChain for building our AI agents

#163

Post by whoknowsidont »

huevosabio wrote: post #109
CharlieDigital wrote: post #2

Bigger problem might be using agents in the first place. We did some testing with agents for content generation (e.g. "authoring" agent, "researcher" agent, "editor" agent) and found that it was easier to just write it as 3 sequential prompts with an explicit control loop. It's easier to debug, monitor, and control the output flow this way. But we still use Semantic Kernel[0] because the lowest level abstractions tha…

What does semantic kernel do for you? It isn't immediately obvious from the Readme.

I'm not OP, but it's just C#/.NET glue and "sample" code for Azure, OpenAI, and a few others (if I were to generously describe it).

It doesn't actually "do" anything or provide useful concepts. I wouldn't use it for anything, personally, even to read.

Re: Why we no longer use LangChain for building our AI agents

#164

Post by mark_l_watson »

I was an early enthusiast of both LangChain and LlamaIndex (and I wrote a book using both frameworks, free to read online [1]) but I had some second thoughts when I started when I started writing LLM examples for my Common Lisp and Racket books that were framework-free, even writing simple vector data stores from scratch. This was, frankly, more fun.

For my personal LLM hacking in Python, I am starting down the same path: writing simple vector data stores in NumPy, write my own prompting tools and LLM wrappers, etc.

I still think that for many developers LangChain and LlamaIndex are very useful (and I try to keep my book up to date), but I usually write about things of most interest to me and I have been thinking of rewriting a new book on framework-free LLM development.

[1] https://leanpub.com/langchain/read

Re: Why we no longer use LangChain for building our AI agents

#165

Post by refulgentis »

Jensson wrote: post #108
refulgentis wrote: post #77

Earlier quoted context omitted.

No. That'd be a good explanation, but it's theoretical. In practice: A) there was no meaningful internal LLM API pre-ChatGPT. All this AI stuff was under lock and key until Nov 2022, then it was an emergency. B) the bits we're discussing are OpenAI-specific concepts that could only have occurred after OpenAI's. The API includes chat messages organized with roles, an OpenAI concept, and "tools", an OpenAI concept, bot…

Google started including LLM features in internal products 2019 at least, I knew since I worked there then. I can't remember exactly when they started having LLM generated snippets and suggestions everywhere but it was there at least since 2019. So they have had internal APIs for this for quite some time. > All this AI stuff was under lock and key until Nov 2022 That is all wrong... Did you work there? What do you ba…

> LLM generated snippets and suggestions everywhere but it was there at least since 2019

Absolutely not. Note that ex. Google's AI answers are not from an LLM and they're very proud of that.

> So they have had internal APIs for this for quite some time.

We did not have internal or external APIs for "chat completions" with chat messages, roles, and JSON schemas until after OpenAI.

> Did you work there?

Yes

> What do you base this on?

The fact it was under lock and key. You had to jump through several layers of approvals to even get access to a standard text-completion GUI, never mind API.

> has been experimenting with LLMs internally ever since the original paper,

What's "the original paper"? Are you calling BERT an LLM? Do you think transformers implied "chat completions"?

> that would make them want different things in their public API as well.

It's a nice theoretical argument.

If you're still convinced Google had a conversational LLM API before OpenAI, or that we need to quibble everything because I might be implying Google didn't invent transformers, there's a much more damning thing:

The API is Gemini-specific and released with Gemini, ~December 2023. There's no reason for it to be so different other than NIH and proto-based thinking. It's not great. That's why ex. we see the other comment where Cloud built out a whole other API and framework that can be used with OpenAI's Python library.

Re: Why we no longer use LangChain for building our AI agents

#166

Post by refulgentis »

DannyBee wrote: post #129
refulgentis wrote: post #77

Earlier quoted context omitted.

No. That'd be a good explanation, but it's theoretical. In practice: A) there was no meaningful internal LLM API pre-ChatGPT. All this AI stuff was under lock and key until Nov 2022, then it was an emergency. B) the bits we're discussing are OpenAI-specific concepts that could only have occurred after OpenAI's. The API includes chat messages organized with roles, an OpenAI concept, and "tools", an OpenAI concept, bot…

>All this AI stuff was under lock and key until Nov 2022, then it was an emergency. This is absolutely false, as the other person said. As one example: We had already built and were using AI based code completion in production by then. Here's a public blog post from July, 2022: https://research.google/blog/ml-enhanced-code-completion-imp... This is just one easy publicly verifiable example, there are others. (We actu…

Pretending that was an LLM as it is understood today, and that whatever internal API was available for internal use cases is actually the same as the public API for Gemini today, and that it was the same as an API for adding a "chat completion" to a "conversation" with messages, roles, and JSON schemas is silly.

Re: Why we no longer use LangChain for building our AI agents

#167

Post by randomdata »

pietro72ohboy wrote: post #161
shpx wrote: post #135

Earlier quoted context omitted.

People call the first actually useful thing the first thing, that's not surprising or wrong.

That statement is patently incorrect. While the 'usefulness' of something can be subjective, the date of creation is an absolute, immutable fact.

What you have failed to grasp is that people are not logic machines. "First chatbot" is never uttered to mean the absolute first chatbot – for all they know someone created an undocumented chatbot in 10,000 B.C. that was lost to time – but merely the first chatbot they are aware of.

Normally the listener is able to read between the lines, but I suppose there may be some defective units out there.

Re: Why we no longer use LangChain for building our AI agents

#168

Post by ironhaven »

moffkalast wrote: post #157
netdevnet wrote: post #152

Earlier quoted context omitted.

You are saying that after having experienced all the subsequent versions. GPT-2 was fairly good, not impressive but fairly good. People were using for all sorts of stuff for the fun of it. The GPT 3 versions were really impressive and had everyone here super excited

I'd argue the GPT-3 results were really cherry picked by the few people who had access, at least if the old versions of 3.5 and turbo are anything to go by. The hype would've died instantly if anyone had actually tried them themselves and realized that there's no consistency. If you want to try out GPT-2 to refresh your memory, here [0] is an online demo. It's bad, I'd say worse than classical graph/tree based autoco…

Open AI when they gave press access to gpt said that you must not publish the raw output for AI safety reasons. So naturally people self selected the best outputs to share.

Re: Why we no longer use LangChain for building our AI agents

#170

Post by randomdata »

dartos wrote: post #138
9dev wrote: post #137

Earlier quoted context omitted.

Well. I'm working on a product that relies on both AI assistants in the user-facing parts, as well as LLM inference in the data processing pipeline. If we let our LLM guy run free, he would create an inscrutable tangled mess of Python code, notebooks, Celery tasks, and expensive VMs in the cloud. I know Pythonista's regard themselves more as artists than engineers, but the rest of us needs reliable and deterministica…

Sounds like your LLM guy just isn’t very good. Langchain is, when you boil it down, an abstraction over text concatenation, staged calls to open ai, and calls to vector search libraries. Even without standard tooling, an experienced programmer should be able to write an understandable system that does those things.

> Sounds like your LLM guy just isn’t very good.

That's the central idea here. Most guys available to hire aren't. Hence why they get constrained into a framework that limits the damage they can cause. In other areas of software development the frameworks are quite mature at this point so it works well enough.

This AI/LLM/whatever you want to call it area of development, however, hadn't garnered much interest until recently, and thus there isn't much in the way of frameworks to lean on. But business is trying to ramp up around it, thus needing to hire those who aren't good to fill seats. Like the parent says, LangChain may not be the framework we want, but it is the one we have, which beats letting the not-very-good developers create some unconstrained mess.

If you win the lottery by snagging one of the small few good developers out there, then certainly you can let them run wild engineering a much better solution. But not everyone is so fortunate.

Post reply on HN