Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we no longer use LangChain for building our AI agents

octomind.dev

Post reply on HN
Print view View original

291–300 of 307 posts

Re: Why we no longer use LangChain for building our AI agents

#291

Post by authorfly »

mikeqq2024 wrote: post #266
authorfly wrote: post #182

Earlier quoted context omitted.

The main reason ChatGPT took off was: 1) Response time of the API of that quality was 10x quicker than the Davinci-instruct-3 model that was released in summer 2022, making interaction more feasible with lower wait times and with concurrency 2) OpenAI strictly banned chat applications on the GPT API; even summarising with more than 150 tokens required your to submit a use case for review; I built an app around this i…

"Instruct tuning was far more impactful than conversational model tuning because instruct enabled so many synthesizing use cases beyond the training data." I saw many model providers nowadays provide instruct model in name as chat model. What difference between instruct tuning and conversational model tuning specifically?

The best papers to read are the T5 paper which introduced intstruction training.

BERT showed that training with two tasks (next sentence and mask fill) was more effective than solely one task.

T5 showed that multiple instructions could be used for one task (token prediction) like not just translating, but also summarizing. They suggested this could generalize (it did)

GPT-2 showed with just token prediction and no instructions you could represent good text; GPT-3 showed this was coherent and also that sufficient context was reliably continued by models(and impacted by the format of training data, e.g. StackOverflow used Q: A: in the training data, so prompts using Q: and A: worked very well for conversation-mimicking).

Davinci-instruct essentially made GPT-3 outputs reliable, because they "corrected model outputs" not just to follow the implicit continued context but to follow text instructions with general english in the users submitted prompt. They could change this to always follow a chat format (e.g. use Pronouns and refer to the user with "You") which seems to work more naturally, but the original instruct worked based on simple commands which are responded to without the chat format (e.g. no "I am sorry" - just no token, no "I believe the book you are looking for is:") etc.

Nowadays most instruct models do actually use prompt formats and training datasets which are conversational (check out the various formats in LM studio) anyway, so the difference is lost.

Re: Why we no longer use LangChain for building our AI agents

#292

Post by randomdata »

manquer wrote: post #281
randomdata wrote: post #78

Earlier quoted context omitted.

SQL (as an API) and databases are orthogonal, though. In fact, I work on an application that uses SQL for its API even though it doesn't even have a database directly attached. All of its data comes from third-parties that only make the data available using REST services. In theory, the app servers that sit in front of those databases could just as easily use SQL instead of GraphQL. Even practically: The libraries ar…

I didn’t say it was a good idea , just elaborated that it is possible. Even if it was easy and solved it all the things say GraphQL does it is still a bad idea . Scaling app servers is relatively easy especially if stateless and follow some of the 12f principles, scaling SQL server horizontally is hard. Multi master , partitioning, sharding even indexing very large tables , de-normalization is ripe with pitfalls and…

Of course it's possible. But you are not beholden to using a database server to use SQL. The app servers can speak SQL too. And, in some cases, should. It is sometimes the right tool for the job.

But if GraphQL is a good fit for your situation, SQL is not. Aside from both enabling ad-hoc execution, there is little overlap between them. They are designed to solve different problems.

Re: Why we no longer use LangChain for building our AI agents

#293

Post by danenania »

siva7 wrote: post #289
danenania wrote: post #222

Earlier quoted context omitted.

I haven't used LangChain, but my sense is that much of what it's really helping people with is stream handling and async control flow. While there are libraries that make it easier, I think doing this stuff right in Python can feel like swimming against the current given its history as a primarily synchronous, single-threaded runtime. I built an agent-based AI coding tool in Go ( https://github.com/plandex-ai/plandex…

Not really. There isn't much that langchain is doing in this regard. The heavy lifting is already done by the original libs like from openai which they use and the rest are just wrappers around their api calls.

The langchain apis seem to include a lot of functionality for ‘chaining’ LLM calls and streams. That’s what I was referring to.

Re: Why we no longer use LangChain for building our AI agents

#294

Post by jackmpcollins »

w4 wrote: post #200
sc077y wrote: post #186

Damn I built a RAG agent during the past 3 months and a half for my internship. And literally everyone in my company was asking me why I wasn't using llangchain or llamaindex like I was a lunatic. Everyone else that built a rag in my company used llangchain, one even went into prod. I kept telling them that it works well if you have a standard usage case but the second you need to something a little original you have…

I had a similar experience when LangChain first came out. I spent a good amount of time trying to use it - including making some contributions to add functionality I needed - but ultimately dropped it. It made my head hurt. Most LLM applications require nothing more than string handling, API calls, loops, and maybe a vector DB if you're doing RAG. You don't need several layers of abstraction and a bucketload of depen…

I completely agree, and built magentic [0] to cover the common needs (structured output, common abstraction across LLM providers, LLM-assisted retries) while leaving all the prompts up to the package user.

[0] https://github.com/jackmpcollins/magentic

Re: Why we no longer use LangChain for building our AI agents

#296

Post by mikeqq2024 »

causal wrote: post #227
ctxc wrote: post #220

Earlier quoted context omitted.

They were early to the scene, made the decisions that made sense at each point in time. Initially I (like many other engineers with no AI exposure) didn't know enough to want to play around with the knobs too much. Now I do. So the playing field has and is changing, langChain are adapting. Isn't that a bit too extreme? Goodwill burnt up? When the field changes, there will be new abstractions - of course I'll have to…

You seem to be implying all abstractions are equal, its just use-case dependent. I disagree- some really are worse than others. In your webdev example, it would not be hard to contrive a framework designed to frustrate. This can also happen by accident. Sometimes bad products really do waste time. In the case of LangChain, I think it was an earnest attempt, but a misguided one. So I'm grateful for LangChain's attempt…

What alternatives are you tring and how good/bad about those?

Re: Why we no longer use LangChain for building our AI agents

#297

Post by mikeqq2024 »

wizzwizz4 wrote: post #35
OutOfHere wrote: post #19

Earlier quoted context omitted.

GraphQL actually holds value in my view as it gives custom SQL-like functionality instead of basic JSON APIs. With it, you can do fewer calls and retrieve only the attributes you need. Granted, if SQL were directly an API, then GraphQL wouldn't hold too much value. Langchain has no such benefit.

> if SQL were directly an API Isn't that what SQL/CLI is for? https://publications.opengroup.org/c451

Sql IS an API, as in android programming, you just use sql as an API to access SQLite, literally.

Re: Why we no longer use LangChain for building our AI agents

#298

Post by mikeqq2024 »

bastawhiz wrote: post #228

Genuine question: can someone point me to a use case where langchain makes the problem easier to solve than using the openai/anthropic/ollama SDKs directly? I've gotten a lot of advice to use langchain, but the docs haven't really shown me how it simplifies the task, or at least not more than using an SDK directly. I really want to at least understand when to use this as a tool but so far I've been failing to figure…

Mind tell what kind of scenario you are tring to solve?

Re: Why we no longer use LangChain for building our AI agents

#299

Post by bastawhiz »

mikeqq2024 wrote: post #298
bastawhiz wrote: post #228

Genuine question: can someone point me to a use case where langchain makes the problem easier to solve than using the openai/anthropic/ollama SDKs directly? I've gotten a lot of advice to use langchain, but the docs haven't really shown me how it simplifies the task, or at least not more than using an SDK directly. I really want to at least understand when to use this as a tool but so far I've been failing to figure…

Mind tell what kind of scenario you are tring to solve?

I literally just want to know what use cases langchain serves. I've built four or five different applications at this point, and it was easy to enough to use various SDKs. Where does langchain come in?
Post reply on HN