Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Langchain Is Pointless

old.reddit.com

Post reply on HN
Print view View original

91–100 of 192 posts

Re: Langchain Is Pointless

#91

Post by Reticularas »

codeptualize wrote: post #72
saulpw wrote: post #64

Earlier quoted context omitted.

I agree, and that's why I've been working on AIPL[0]. Our first v0.1 release should be in the next few days. https://github.com/saulpw/aipl It's basically just a simple scripting language with array semantics and inline prompt construction, and you can drop into Python any time you like.

With peace and love, why would I not just write some code? LLM pipelines are not very complex: it's string manipulation, api calls, and storage, there is not much more too it. All are quite easy to do, often needing nothing else but the standard library. For more complex cases or bigger scale you have a plethora of battle tested solutions to manage things like queues, back off and retries, concurrency, etc. Maybe it'…

I get this feeling too "maybe I'm not the target audience" This feeling is followed closely by "Who is the target audience?". Some abstract concept of an audience doing complex LLM work to accomplish... something?

The value in statistics analysis for LLMs is clear, the value in chaining responses is very unclear.

Re: Langchain Is Pointless

#92

Post by space_fountain »

LASR wrote: post #85

The reason why Langchain is pointless is that it's trying to solve problems on top of technical foundations that just cannot support it. The #1 learning is that there is no reusability with the current generation of LLMs. We're using GPT-4 and 3.5T exclusively. Over the last several months, my team has been building several features using highly sophisticated LLM chains that do all manner of reasoning. The ultimate o…

How do you deal with the prompt iteration phase and how coupled is that to the DAG phase? I've only worked on a few proofs of concept in this phase, but a thing I struggled with was a strong desire to allow non technical colleagues to mess with the prompts. It wasn't clear to me how much the prompts need to evolve in tandem with the the DAG and how much they can exist separately

Re: Langchain Is Pointless

#93

Post by earleybird »

m3kw9 wrote: post #34

Langchain is as pointless as one of those kitchen tools that specifically cuts egg into slices, but you could have done that with a knife.

I like that analogy. I have one of those slicers and every time I use it I have two thoughts. First, this slices so quick and evenly. Second, there's a whole lot more I have to clean up compared to the knife.

Re: Langchain Is Pointless

#95

Post by moneywoes »

LASR wrote: post #85

The reason why Langchain is pointless is that it's trying to solve problems on top of technical foundations that just cannot support it. The #1 learning is that there is no reusability with the current generation of LLMs. We're using GPT-4 and 3.5T exclusively. Over the last several months, my team has been building several features using highly sophisticated LLM chains that do all manner of reasoning. The ultimate o…

In that case what pattern do you use for integrations?

Re: Langchain Is Pointless

#96

Post by mmq »

jpulec wrote: post #87
mmq wrote: post #80

Earlier quoted context omitted.

> When OpenAI released GPT-4, we were able to change one parameter, and everything still just worked. Wouldn't that be the same if you used the OAI js library directly? Basically swapping the model parameter?

It's not. The API is different, since GPT-4 is a chat based model, and davinci isn't. It's not a huge difference, but these little sort of things add up.

I see, thought you were using GPT-3.5 and moved to GPT-4.

Re: Langchain Is Pointless

#97

Post by redox99 »

teaearlgraycold wrote: post #36
hospitalJail wrote: post #21

Does it introduce more lines of code? Yes Does it introduce features that you don't need to implement yourself? Yes Does it make it easy to drop OpenAI? Yes What's the problem?

You'll still need to re-tune your prompt and the hyperparameters when switching models. So the actual effort of switching models is not improved much if at all.

Hopefully as LLMs get smarter, you'll need less prompt engineering and stuff will "just work" across models.

Re: Langchain Is Pointless

#98

Post by krychu »

Using an LLM framework at this moment doesn’t make sense and can be damaging, in my humble opinion. Ways to extract value from LLMs are in early exploration stage. Look at research in prompting: chain of thought, react, reflection, tree of thoughts, zero vs few hot etc. Then completion vs conversational interfacing. Then memory management via vector databases and prompt expansion vs compression vs progressive summarization etc. All these are fairly recent developments. They are not abstractions worth cementing, this is search and creative phase. LLMs threw everything in the air, but the dust is far from settling. I think it’s important to recognize the phase we’re in and pick your weapon accordingly. You have to stay nimble and light, ready to experiment with a new idea that will come out next week. You should be hacking these things together by yourself. If you pick a framework at this stage know that the framework will have to pay the price of trying to cement things in the times of storm. And you’ll be a few steps behind. Of course this is my personal take.

Re: Langchain Is Pointless

#99

Post by LASR »

space_fountain wrote: post #92
LASR wrote: post #85

The reason why Langchain is pointless is that it's trying to solve problems on top of technical foundations that just cannot support it. The #1 learning is that there is no reusability with the current generation of LLMs. We're using GPT-4 and 3.5T exclusively. Over the last several months, my team has been building several features using highly sophisticated LLM chains that do all manner of reasoning. The ultimate o…

How do you deal with the prompt iteration phase and how coupled is that to the DAG phase? I've only worked on a few proofs of concept in this phase, but a thing I struggled with was a strong desire to allow non technical colleagues to mess with the prompts. It wasn't clear to me how much the prompts need to evolve in tandem with the the DAG and how much they can exist separately

There are a few increasingly harder things when it comes to prompt customization:

1. Prompts ask LLM to generate input for the next step

2. Prompts ask LLM to generate instructions for the next step

3. Prompts ask LLM to generate the next step

Doing #3 across multiple steps is the promise of Langchain, AutoGPT et al. Pretty much impossible to do with useful quality. Attempting to do #3 very often either ends up completing the chain too early, or just spinning in a loop. Not the kind of thing you can optimize iteratively to good enough quality at production scale. "Retry" as a user-facing operation is just stupid IMO. Either it works well, or we don't offer it as a feature.

So we stopped doing 3 completely. The features now have a narrow usecase and a fully-defined DAG shape upfront. We feed some context on what all the steps are to every step, so it can understand the overall purpose.

#2, we tune these prompts internally within the team. It's very sensitive to specific words. Even things like newlines affects quality too much.

#1 - we've found it's doable for non-tech folks. In some of the features, we expose this to the user somewhat as additional context and mix that in with the pre-built instructions.

So #2 is where it's both hard to get right and still solvable. Every prompt change has to be tested with a huge number of full-chain invocations on real input data before it can be accepted and stabilized. The evaluation of quality is all human, manual work. We tried some other semi-automated approaches, but just not feasible.

All of this is why there is no way Langchain or anything like it is currently useful to built actually valuable user-facing features at production scale.

Re: Langchain Is Pointless

#100

Post by teaearlgraycold »

LASR wrote: post #85

The reason why Langchain is pointless is that it's trying to solve problems on top of technical foundations that just cannot support it. The #1 learning is that there is no reusability with the current generation of LLMs. We're using GPT-4 and 3.5T exclusively. Over the last several months, my team has been building several features using highly sophisticated LLM chains that do all manner of reasoning. The ultimate o…

100% agreed. I've used GPT professionally and we would try out different hosts, AI21, etc. and it there were always clear quality issues with just re-using your prompt and hyperparameters. Some of that was down to other models being lesser quality, but we'd also need to re-tune prompts when upgrading to new OpenAI models for the best effect. It turns out that LLMs aren't quite a commodity.
Post reply on HN