Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM that can call multiple tool APIs with one request

cohere.com

Post reply on HN
Print view View original

61–69 of 69 posts

Re: LLM that can call multiple tool APIs with one request

#61

Post by lnrd »

danw1979 wrote: post #55

It feels like the industry is heading towards automating everything using non-deterministic black boxes stuck together with proprietary glue.

It's something that we know it will backfire very spectacularly, but might happen anyway. I could see this backfire and have a industry wide reversal to other technologies. I have a gut feeling that already happened in other areas, like companies going back to bare-metal, but it's not really the best example sine cloud was and still is the best solution for most companies.

Does anyone with more experience than me have memories of similar things happening? Where a technology was hyped and adopted anywhere until something happened that caused an industry-wide reversal to more established ways of doing things?

Re: LLM that can call multiple tool APIs with one request

#62

Post by openmajestic »

TZubiri wrote: post #7

I do this for a living, Ask Me Anything. Before they were called tools they were called function calls in ChatGpt. Before that we had response_format = "json_object" And even before that we were prompting with function signatures and asking it to output parameters.

It was called tool use before ChatGpt existed.

Re: LLM that can call multiple tool APIs with one request

#63

Post by bhl »

TZubiri wrote: post #7

I do this for a living, Ask Me Anything. Before they were called tools they were called function calls in ChatGpt. Before that we had response_format = "json_object" And even before that we were prompting with function signatures and asking it to output parameters.

What’s the difference between tool calling and “agents”?

How would you handle map-reduce type of tool calls where you have a lot of parallel tools that you want to merge later on? What’s a good way to scale that without running into API limits?

Re: LLM that can call multiple tool APIs with one request

#64

Post by TZubiri »

verdverm wrote: post #41
TZubiri wrote: post #37

Earlier quoted context omitted.

btw that's not how tools work at all. Tools are function/API based. (Unless you expose a function run_sql(query), but that's on you.)

I brought it up because popular frameworks are offering this type of agent or function out of the box There is no "way that tools work" You pass OpenAPI like schemas along with the prompt and you get back a JSON object. The rest is code and you can do anything you want with. The LLM is merely mapping from unstructured text onto a schema best it can, and we know they are imperfect.

https://en.wikipedia.org/wiki/Robustness_principle

"be conservative in what you send, be liberal in what you accept"

LLM parses text into a list of parameters. You design your function such that it is safe regardless of what the parameters are.

Re: LLM that can call multiple tool APIs with one request

#65

Post by TZubiri »

verdverm wrote: post #23
TZubiri wrote: post #21

Earlier quoted context omitted.

I personally don't connect LLMs to SQL, but to APIs. But I'm pretty sure you would just give an SQL user to the LLM and enjoy the SQL server's built-in permissions and auditing features.

What if that user has write permissions and the LLM generates a bad UPDATE, i.e. forgets to put the WHERE clause in... even for a SELECT, how do you know the right constraints were in place and you are getting the correct data? read-only use-cases misses a whole category. All this is to get back to the point that people want to audit the LLM before running the function because of the unreliability, there is hesitance…

How about stored procedures for Write operations?

Re: LLM that can call multiple tool APIs with one request

#66

Post by etse »

OutOfHere wrote: post #54
etse wrote: post #53

Earlier quoted context omitted.

If you wanted to compare OpenAI models against Anthropic or Google, wouldn't the framework help a lot? Breaking APIs is more about bad framework development than frameworks in general. I think frameworks tend to provide an escape hatch. LlamaIndex comes to mind. It seems to me that by not learning and using an existing framework, you're building your own, which is a calculated tradeoff.

That is a good use case and it's a good problem to have, certainly the kind I wanted to hear, but it's not a problem I have had yet. Moreover, I absolutely expect to have to update my prompts if I have to support a different model, even if its a different model by the same provider. For example, there is a difference in the behavior of gpt4-turbo vs gpt4-o even though both are by OpenAI. Specific LLMs have specific t…

I'm not quite understanding how different prompts for different models reduces the attractiveness of a framework. A framework could theoretically have an LLM evals package to run continuous experiments of all prompts against across all models.

Also theoretically, an LLM framework could estimate costs, count tokens, offer a variety of chunking strategies, unify the more sophisticated APIs, like tools or agents–all of which could vary from provider to provider.

Admittedly, this view came just from doing early product explorations, but a framework was helpful for most of the above reasons (I didn't find an evals framework that I liked).

You mentioned not having this problem yet. What kind of problems have you been running across? I'm wondering if I'm missing some other context.

Re: LLM that can call multiple tool APIs with one request

#68

Post by madrox »

patleeman wrote: post #58
madrox wrote: post #2

I have a saying: "any sufficiently advanced agent is indistinguishable from a DSL" If I'm really leaning into multi-tool use for anything resembling a mutation, then I'd like to see an execution plan first. In my experience, asking an AI to code up a script that calls some functions with the same signature as tools and then executing that script actually ends up being more accurate than asking it to internalize its a…

Very curious if anybody else is working on a DSL that's meant for LLMs to output? Has anyone seen anyone using this approach? Any resources available?

Funny enough, there's a DSL LLMs already tend to be trained on: JavaScript. Just teach the LLM the function signatures it has to work with, then execute the script in a VM

Re: LLM that can call multiple tool APIs with one request

#69

Post by RecycledEle »

patleeman wrote: post #57
RecycledEle wrote: post #44

Earlier quoted context omitted.

> I have a saying: "any sufficiently advanced agent is indistinguishable from a DSL" I don't think you mean Digital Subscriber Line, so may I ask: What is a DSL in this context?

Domain Specific Language

Thank you.
Post reply on HN