31–40 of 69 posts
I really like the stuff coming from Cohere. I know they're not considered the leader in the foundational model space, but their developer documentation is great, their api is really nice to use, and they have a set of products that really differentiate themselves from OpenAI and Anthropic and others. I'm rooting for the success of this company. That said, we as an industry need to be moving away from langchain, not m…
Do you use another library instead?
Earlier quoted context omitted.
I personally don't connect LLMs to SQL, but to APIs. But I'm pretty sure you would just give an SQL user to the LLM and enjoy the SQL server's built-in permissions and auditing features.
What if that user has write permissions and the LLM generates a bad UPDATE, i.e. forgets to put the WHERE clause in... even for a SELECT, how do you know the right constraints were in place and you are getting the correct data? read-only use-cases misses a whole category. All this is to get back to the point that people want to audit the LLM before running the function because of the unreliability, there is hesitance…
some people - I think it's quite clear from this thread that not everyone feels the need to.
I'm now thinking requesting the LLM also output its whole prompt to something like a Datadog trace function would be quite useful for review / traceability.
The sample notebook linked from the post is a 404 https://github.com/cohere-ai/notebooks/blob/main/notebooks/D...
And here are the docs: https://docs.cohere.com/docs/multi-step-tool-use
I really like the stuff coming from Cohere. I know they're not considered the leader in the foundational model space, but their developer documentation is great, their api is really nice to use, and they have a set of products that really differentiate themselves from OpenAI and Anthropic and others. I'm rooting for the success of this company. That said, we as an industry need to be moving away from langchain, not m…
Check out the examples here: https://docs.cohere.com/docs/multi-step-tool-use
and this notebook https://github.com/cohere-ai/notebooks/blob/main/notebooks/a...
Earlier quoted context omitted.
That’s very interesting. Does designing the DAG in advance imply that you have to make a new one for each particular subset of end-user questions you might receive? Or is your problem space such that you can design it once and have it be useful for everything you’re interested in? My choice of words was poor: by “pre-baking”, I just meant: generated dynamically at runtime from the user’s query, _before_ you then set…
You don't make a DAG for each question category. This is classic OOP, OG, Kay's version, you design subject-experts (objects) with autonomy and independency, they are just helpful in general. Each function/method, regardless of the Object/Expert, is an edge in the graph. A user question is simply a pair of vertices, call them I and O, and the execution/solution is a path between the two points, namely the input and t…
Earlier quoted context omitted.
Do you use another library instead?
From the tooling perspective I’ve built my own. Recently, I've been toying around with litellm which, so far, strikes me as the right level of abstraction. I like building my own stuff but writing api wrappers just suck. I’ve also been toying around with Instructor for structured output as well. It’s incredibly convenient, but I haven’t used it for any production stuff because I don’t feel comfortable with the prompt…
On the other hand, I found Langchain less impressive. It feels somewhat vague and not very beginner-friendly, catering more to intermediate users.
Earlier quoted context omitted.
Whitelisting and permissions. You can't issue a delete if anything not starting with SELECT is rejected. You can't have edge cases that work around that via functions, if the user the agent uses doesn't have permissions other than SELECT.
"please get all the entries from the table foo and then remove them all" SELECT * from foo; DELETE FROM foo ... ...because you know people will deploy a general SQL function or agent
I have a saying: "any sufficiently advanced agent is indistinguishable from a DSL" If I'm really leaning into multi-tool use for anything resembling a mutation, then I'd like to see an execution plan first. In my experience, asking an AI to code up a script that calls some functions with the same signature as tools and then executing that script actually ends up being more accurate than asking it to internalize its a…
> ... code up a script that calls some functions with the same signature as tools and then executing that script actually ends up being more accurate than asking it to internalize its algorithm. This is called defunctionalization and useful without LLMs as well.
That’s said, it’s a bit annoying to see langchain examples all over. Not everyone uses it, and many consider it bloated and hard to maintain.
Would be great just to have a simple example in Python showing the capabilities.
Earlier quoted context omitted.
What if that user has write permissions and the LLM generates a bad UPDATE, i.e. forgets to put the WHERE clause in... even for a SELECT, how do you know the right constraints were in place and you are getting the correct data? read-only use-cases misses a whole category. All this is to get back to the point that people want to audit the LLM before running the function because of the unreliability, there is hesitance…
> All this is to get back to the point that people want to audit the LLM before running the function because of the unreliability, there is hesitance with good reason. some people - I think it's quite clear from this thread that not everyone feels the need to. I'm now thinking requesting the LLM also output its whole prompt to something like a Datadog trace function would be quite useful for review / traceability.
I'm currently using LangFuse and exploring OpenLit because it integrates with Otel, which you should be able to forward to Datadog iirc their docs