Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM that can call multiple tool APIs with one request

cohere.com

Post reply on HN
Print view View original

51–60 of 69 posts

Re: LLM that can call multiple tool APIs with one request

#51

Post by el-ai-ne »

Oras wrote: post #39

Cohere is underrated, I recently tried cohere R model and found it following prompt much better than gpt-4o and even Claude opus. That’s said, it’s a bit annoying to see langchain examples all over. Not everyone uses it, and many consider it bloated and hard to maintain. Would be great just to have a simple example in Python showing the capabilities.

Hey, I work at Cohere and I'm stoked to hear Command R is following prompt instructions well. Appreciate the feedback and here is a simple example in Python: https://docs.cohere.com/docs/multi-step-tool-use#using-the-c...

The following cookbooks contain slightly more advanced code examples, using just the cohere API for multi-step: https://docs.cohere.com/page/calendar-agent https://docs.cohere.com/page/pdf-extractor https://docs.cohere.com/page/agentic-multi-stage-rag

Cheers

Re: LLM that can call multiple tool APIs with one request

#52

Post by OutOfHere »

SeriousStorm wrote: post #50
OutOfHere wrote: post #48

I have developed multiple multi-step LLM workflows, expressible as both conditional and parallel DAGs, using mostly plain Python, and I still don't understand why these langchain-type libraries feel the need to exist. Plain Python is quite sufficient for advanced LLM workflows if you know how to use it. LLMs are innately unreliable, and they require a lot of hand-holding and prompt-tuning to get them to work well. Ge…

Every time I look into building a workflow with langchain it seems unnecessarily complex. So I end up stopping. Are you just running an LLM server (Ollama, llama.cpp, etc) and then making API calls to that server with plain Python or is it more than that?

I suppose ollama and llama.cpp, or at least any corresponding Python SDKs, would be good for using self-hosted models, especially if they support parallel GPU use. If it's something custom, Pytorch would come into the picture. In production workflows, it can obviously be useful to run certain LLM prompts in parallel to hasten the job.

For now I have used only cloud APIs with their Python SDKs, including the prompt completion, TTS, and embedding endpoints. They allow me to run many jobs in parallel which is useful for complex workflows or if facing heavy user demand. For caching of responses, I have used a local disk caching library, although I guess one can alternatively use a standalone or embedded database. I have used threading via `concurrent.futures` for concurrent jobs, although asyncio too would work.

The one simple external Python library I found so far is `semantic-text-splitter` for splitting long texts using token counts, but this too I could have done by myself with a bit of effort. I think langchain has something for it too.

Re: LLM that can call multiple tool APIs with one request

#53

Post by etse »

OutOfHere wrote: post #48

I have developed multiple multi-step LLM workflows, expressible as both conditional and parallel DAGs, using mostly plain Python, and I still don't understand why these langchain-type libraries feel the need to exist. Plain Python is quite sufficient for advanced LLM workflows if you know how to use it. LLMs are innately unreliable, and they require a lot of hand-holding and prompt-tuning to get them to work well. Ge…

If you wanted to compare OpenAI models against Anthropic or Google, wouldn't the framework help a lot? Breaking APIs is more about bad framework development than frameworks in general.

I think frameworks tend to provide an escape hatch. LlamaIndex comes to mind. It seems to me that by not learning and using an existing framework, you're building your own, which is a calculated tradeoff.

Re: LLM that can call multiple tool APIs with one request

#54

Post by OutOfHere »

etse wrote: post #53
OutOfHere wrote: post #48

I have developed multiple multi-step LLM workflows, expressible as both conditional and parallel DAGs, using mostly plain Python, and I still don't understand why these langchain-type libraries feel the need to exist. Plain Python is quite sufficient for advanced LLM workflows if you know how to use it. LLMs are innately unreliable, and they require a lot of hand-holding and prompt-tuning to get them to work well. Ge…

If you wanted to compare OpenAI models against Anthropic or Google, wouldn't the framework help a lot? Breaking APIs is more about bad framework development than frameworks in general. I think frameworks tend to provide an escape hatch. LlamaIndex comes to mind. It seems to me that by not learning and using an existing framework, you're building your own, which is a calculated tradeoff.

That is a good use case and it's a good problem to have, certainly the kind I wanted to hear, but it's not a problem I have had yet.

Moreover, I absolutely expect to have to update my prompts if I have to support a different model, even if its a different model by the same provider. For example, there is a difference in the behavior of gpt4-turbo vs gpt4-o even though both are by OpenAI.

Specific LLMs have specific tendencies and preferences which one has to work with. What I'm saying is that the framework will help, but it's not as simple as switching the model class.

Re: LLM that can call multiple tool APIs with one request

#56

Post by TZubiri »

cpursley wrote: post #42
TZubiri wrote: post #7

I do this for a living, Ask Me Anything. Before they were called tools they were called function calls in ChatGpt. Before that we had response_format = "json_object" And even before that we were prompting with function signatures and asking it to output parameters.

Are you working on a product or doing consulting?

Consulting: goldenwordai.com

Product (Unreleased): silverletterai.com

Re: LLM that can call multiple tool APIs with one request

#57

Post by patleeman »

RecycledEle wrote: post #44
madrox wrote: post #2

I have a saying: "any sufficiently advanced agent is indistinguishable from a DSL" If I'm really leaning into multi-tool use for anything resembling a mutation, then I'd like to see an execution plan first. In my experience, asking an AI to code up a script that calls some functions with the same signature as tools and then executing that script actually ends up being more accurate than asking it to internalize its a…

> I have a saying: "any sufficiently advanced agent is indistinguishable from a DSL" I don't think you mean Digital Subscriber Line, so may I ask: What is a DSL in this context?

Domain Specific Language

Re: LLM that can call multiple tool APIs with one request

#58

Post by patleeman »

madrox wrote: post #2

I have a saying: "any sufficiently advanced agent is indistinguishable from a DSL" If I'm really leaning into multi-tool use for anything resembling a mutation, then I'd like to see an execution plan first. In my experience, asking an AI to code up a script that calls some functions with the same signature as tools and then executing that script actually ends up being more accurate than asking it to internalize its a…

Very curious if anybody else is working on a DSL that's meant for LLMs to output?

Has anyone seen anyone using this approach? Any resources available?

Re: LLM that can call multiple tool APIs with one request

#59

Post by TZubiri »

verdverm wrote: post #22
TZubiri wrote: post #20

Earlier quoted context omitted.

That'S not how it works. The user questions are expressed in business-domain language. "give me the names of all the employees and then remove them all" is parsed, maybe as: " employees(), delete(employees())". It's up to the programmer to define the available functions, if employees() is available, then the first result will be provided, if not it won't. If the functoin delete with a list of employees as parameter i…

> That'S not how it works They are actually quite flexible and you can do anything you want. You supply the LLM with the function names and possible args. I can easily define "sql(query: string)" as a flexible SQL function the LLM can use re: permissions, as soon as you have write permissions, you have dangerous potential. LLMs are not reliable enough, nor are humans, which is why we use code review.

Correct, by "it" I was referring specifically to the "Tools" functionality that is the subject of the linked article.

Tools DON'T generate sql queries, they generate function calls. Of course you can hack it to output sql queries, you can always hack something beyond its intended design purpose, but it's not what it's supposed to be doing.

Re: permissions, nothing new here, give your LLM agent the permissions of a Junior DBA.

Re: LLM that can call multiple tool APIs with one request

#60

Post by TZubiri »

verdverm wrote: post #30
TZubiri wrote: post #29

Earlier quoted context omitted.

No, the human user doesn't have permissions, the LLM system has permissions, we create a user for the process, we've been doing this since unix, take a look at what your HTTP server runs as. There's no deputization of permissions going on here, at least on my systems. Even if there are user-level permissions, you then use a role-based approach (SQL user for a type of users, for example accountant, manager, etc..) and…

and even with that permission system, mistakes still happen, we haven't even been able to eliminate sql injection in real systems, so these things can and will happen adding LLMs in means we have an unaudited query producer, that is the point OP is trying to make, that is something they want to avoid and audit the function call before it happens, because we know the LLMs are not even at our level yet, and we make mis…

Again, SQL query generating agents are not the subject of the original article.
Post reply on HN