Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM that can call multiple tool APIs with one request

cohere.com

Post reply on HN
Print view View original

41–50 of 69 posts

Re: LLM that can call multiple tool APIs with one request

#41

Post by verdverm »

TZubiri wrote: post #37
verdverm wrote: post #16

Earlier quoted context omitted.

"please get all the entries from the table foo and then remove them all" SELECT * from foo; DELETE FROM foo ... ...because you know people will deploy a general SQL function or agent

btw that's not how tools work at all. Tools are function/API based. (Unless you expose a function run_sql(query), but that's on you.)

I brought it up because popular frameworks are offering this type of agent or function out of the box

There is no "way that tools work"

You pass OpenAPI like schemas along with the prompt and you get back a JSON object. The rest is code and you can do anything you want with. The LLM is merely mapping from unstructured text onto a schema best it can, and we know they are imperfect.

Re: LLM that can call multiple tool APIs with one request

#42

Post by cpursley »

TZubiri wrote: post #7

I do this for a living, Ask Me Anything. Before they were called tools they were called function calls in ChatGpt. Before that we had response_format = "json_object" And even before that we were prompting with function signatures and asking it to output parameters.

Are you working on a product or doing consulting?

Re: LLM that can call multiple tool APIs with one request

#43

Post by Scipio_Afri »

TZubiri wrote: post #24
darkteflon wrote: post #15

Earlier quoted context omitted.

That’s very interesting. Does designing the DAG in advance imply that you have to make a new one for each particular subset of end-user questions you might receive? Or is your problem space such that you can design it once and have it be useful for everything you’re interested in? My choice of words was poor: by “pre-baking”, I just meant: generated dynamically at runtime from the user’s query, _before_ you then set…

You don't make a DAG for each question category. This is classic OOP, OG, Kay's version, you design subject-experts (objects) with autonomy and independency, they are just helpful in general. Each function/method, regardless of the Object/Expert, is an edge in the graph. A user question is simply a pair of vertices, call them I and O, and the execution/solution is a path between the two points, namely the input and t…

Thanks for your perspective! Can you explain why oral is a bad idea?

Re: LLM that can call multiple tool APIs with one request

#44

Post by RecycledEle »

madrox wrote: post #2

I have a saying: "any sufficiently advanced agent is indistinguishable from a DSL" If I'm really leaning into multi-tool use for anything resembling a mutation, then I'd like to see an execution plan first. In my experience, asking an AI to code up a script that calls some functions with the same signature as tools and then executing that script actually ends up being more accurate than asking it to internalize its a…

> I have a saying: "any sufficiently advanced agent is indistinguishable from a DSL"

I don't think you mean Digital Subscriber Line, so may I ask: What is a DSL in this context?

Re: LLM that can call multiple tool APIs with one request

#45

Post by TZubiri »

Scipio_Afri wrote: post #43
TZubiri wrote: post #24

Earlier quoted context omitted.

You don't make a DAG for each question category. This is classic OOP, OG, Kay's version, you design subject-experts (objects) with autonomy and independency, they are just helpful in general. Each function/method, regardless of the Object/Expert, is an edge in the graph. A user question is simply a pair of vertices, call them I and O, and the execution/solution is a path between the two points, namely the input and t…

Thanks for your perspective! Can you explain why oral is a bad idea?

IO STDs

Re: LLM that can call multiple tool APIs with one request

#46

Post by ai4ever »

madrox wrote: post #2

I have a saying: "any sufficiently advanced agent is indistinguishable from a DSL" If I'm really leaning into multi-tool use for anything resembling a mutation, then I'd like to see an execution plan first. In my experience, asking an AI to code up a script that calls some functions with the same signature as tools and then executing that script actually ends up being more accurate than asking it to internalize its a…

putting the LLM in the loop makes the tooling unreliable. so, the usecases would be limited to those where accuracy is not important.

whereas, a DSL still aims for accurate and deterministic modeling of the specific usecase.

Re: LLM that can call multiple tool APIs with one request

#47

Post by kakaly0403 »

verdverm wrote: post #40
_puk wrote: post #32

Earlier quoted context omitted.

> All this is to get back to the point that people want to audit the LLM before running the function because of the unreliability, there is hesitance with good reason. some people - I think it's quite clear from this thread that not everyone feels the need to. I'm now thinking requesting the LLM also output its whole prompt to something like a Datadog trace function would be quite useful for review / traceability.

Most LLM observability tools do this I'm currently using LangFuse and exploring OpenLit because it integrates with Otel, which you should be able to forward to Datadog iirc their docs

Checkout Langtrace. It’s also OTEL and integrates with datadog.

https://github.com/Scale3-Labs/langtrace

Re: LLM that can call multiple tool APIs with one request

#48

Post by OutOfHere »

I have developed multiple multi-step LLM workflows, expressible as both conditional and parallel DAGs, using mostly plain Python, and I still don't understand why these langchain-type libraries feel the need to exist. Plain Python is quite sufficient for advanced LLM workflows if you know how to use it.

LLMs are innately unreliable, and they require a lot of hand-holding and prompt-tuning to get them to work well. Getting into the low-level details of the prompts is too essential. I don't want any libraries to come in the way because I have to be able to find and cleverly prevent the failure cases that happen just 1 in 500 times.

These libraries seem to mainly just advertise each other. If I am missing something, I don't know what it is.

Re: LLM that can call multiple tool APIs with one request

#49

Post by leobg »

OutOfHere wrote: post #48

I have developed multiple multi-step LLM workflows, expressible as both conditional and parallel DAGs, using mostly plain Python, and I still don't understand why these langchain-type libraries feel the need to exist. Plain Python is quite sufficient for advanced LLM workflows if you know how to use it. LLMs are innately unreliable, and they require a lot of hand-holding and prompt-tuning to get them to work well. Ge…

Always felt the same way, but could never put it in words as eloquently as you just did. Python (or any other programming language) already is the best glue. With these frameworks, you just waste brain cycles on learning APIs that change and break every couple of months.

Re: LLM that can call multiple tool APIs with one request

#50

Post by SeriousStorm »

OutOfHere wrote: post #48

I have developed multiple multi-step LLM workflows, expressible as both conditional and parallel DAGs, using mostly plain Python, and I still don't understand why these langchain-type libraries feel the need to exist. Plain Python is quite sufficient for advanced LLM workflows if you know how to use it. LLMs are innately unreliable, and they require a lot of hand-holding and prompt-tuning to get them to work well. Ge…

Every time I look into building a workflow with langchain it seems unnecessarily complex. So I end up stopping.

Are you just running an LLM server (Ollama, llama.cpp, etc) and then making API calls to that server with plain Python or is it more than that?

Post reply on HN