Cutting LLM Costs by 83% with LangSmith

#1

Post by nick_rocks » Fri, Oct 27, 2023, 3:52 PM UTC

Cutting LLM Costs by 83% with LangSmith
medium.com

Re: Cutting LLM Costs by 83% with LangSmith

#2

Post by nick_rocks » Fri, Oct 27, 2023, 3:52 PM UTC

83% cost reduction seems impressive, but is this really an issue for anyone?

Re: Cutting LLM Costs by 83% with LangSmith

#3

Post by sp332 » Fri, Oct 27, 2023, 4:16 PM UTC

nick_rocks wrote: post #2

83% cost reduction seems impressive, but is this really an issue for anyone?

Well, I heard of one case where a bug was causing queries of 100,000 tokens at a time. https://medium.com/dataherald/cutting-llm-costs-by-83-with-l...

Re: Cutting LLM Costs by 83% with LangSmith

#4

Post by [deleted] » Fri, Oct 27, 2023, 4:52 PM UTC

sp332 wrote: post #3

nick_rocks wrote: post #2

83% cost reduction seems impressive, but is this really an issue for anyone?

Well, I heard of one case where a bug was causing queries of 100,000 tokens at a time. https://medium.com/dataherald/cutting-llm-costs-by-83-with-l...

[deleted]

Re: Cutting LLM Costs by 83% with LangSmith

#5

Post by nick_rocks » Fri, Oct 27, 2023, 4:53 PM UTC

sp332 wrote: post #3

nick_rocks wrote: post #2

83% cost reduction seems impressive, but is this really an issue for anyone?

Well, I heard of one case where a bug was causing queries of 100,000 tokens at a time. https://medium.com/dataherald/cutting-llm-costs-by-83-with-l...

Fair enough! But have you heard of 2 cases?

Re: Cutting LLM Costs by 83% with LangSmith

#6

Post by aazo11 » Fri, Oct 27, 2023, 5:53 PM UTC

Is langsmith still useful if your app is not built with Langchain? Given how early these technologies wouldn't it be better to use a framework agnostic tool to not be tied in to a single framework?