Earlier quoted context omitted.
confidence all the way down
Confidence is all you need.
True in many situations in life.
Learnings from building AI agents
61–70 of 70 posts
Re: Learnings from building AI agents
#62Funny thing is the structured output in the last example. ``` { "reasoning": "`cfg` can be nil on line 42; dereferenced without check on line 47", "finding": "Possible nil‑pointer dereference", "confidence": 0.81 } ``` You know the confidence value is completely bogus, don't you?
you know everything is made up right? And yet it just works. I too use a confidence score in an bug finder app, Github seems to use them in copilot reviews, people will use them until it is shown not to work anymore.
on the other hand this post https://www.greptile.com/blog/make-llms-shut-up says that it didn't work in their case:
> Sadly, this also failed. The LLMs judgment of its own output was nearly random. This also made the bot extremely slow because there was now a whole new inference call in the workflow.
Re: Learnings from building AI agents
#63Earlier quoted context omitted.
elzbardico is pointing out how the author is having the confidence value generated in the output of the response rather than it being the confidence of the output.
Is there research solid knowledge on this?
this trick is being used by many apps (including Github copilot reviews). The way I see it, is that if the agent has an eager-to-please problem, then you give it a way out
Re: Learnings from building AI agents
#64The problem is that, regardless of how you try to use "micro-agents " as a marketing term, LLMs are instructed to return a result. They will always try to come up with something. The example provided was a poor one. The comment from LLM was solid. Why would you comment out a step in the pipeline instead of just deleting it? I would comment the same in a PR.
By splitting prompts into smaller chunks you effectively get “bias free” opinions, especially when cross-checked. You can then turn them into local reasoning, which is different from “sending an email to the LLM” which seems to be the case here. Remember, LLM is Rainman.
Re: Learnings from building AI agents
#65ah the joy of non-determinism. Have fun tweaking till you die. Also I wish youa lot of fun giving your customers buttons to disable/enable options.
Re: Learnings from building AI agents
#66Earlier quoted context omitted.
Is there research solid knowledge on this?
this trick is being used by many apps (including Github copilot reviews). The way I see it, is that if the agent has an eager-to-please problem, then you give it a way out
Thanks. I was talking about the confidence measure.
Re: Learnings from building AI agents
#67Earlier quoted context omitted.
Otherwise known as science 1:Observation 2:Hypothesis 3:test 4:GOTO:1 This is every thing ever built ever What is the problem exactly?
For one thing, what you learned can stop working when you switch to a new model, or just a newer version of the “same” model.
All that means is that you verified the null hypothesis which should be that it doesn’t work
If you create hypothesis tests that are not written in or specific enough then you’re right you’re not gonna be able to do science
Incidentally 99.9% of people I know have no instinct for how to actually do science or have rigor or focus to actually do it in a way that is usable
Re: Learnings from building AI agents
#68Funny thing is the structured output in the last example. ``` { "reasoning": "`cfg` can be nil on line 42; dereferenced without check on line 47", "finding": "Possible nil‑pointer dereference", "confidence": 0.81 } ``` You know the confidence value is completely bogus, don't you?
Easy fix, just have the LLM generate: { "reasoning": "`cfg` can be nil on line 42; dereferenced without check on line 47", "finding": "Possible nil‑pointer dereference", "confidence": 0.81, "confidence_in_confidence_rating": 0.54, "confidence_in_confidence_rating_in_confidence_rating": 0.12, "confidence_in_confidence_rating_in_confidence_rating_in_confidence_rating": 0.98, // Etc... }
Wasteful. `confidence`'s type should be Array, wherein confidence[N] gives the Nth derivative confidence rating.
Re: Learnings from building AI agents
#69Earlier quoted context omitted.
I've found that giving agents an "opt out" works pretty well. For structured outputs, making fields optional isn't usually enough. Providing an additional field for it to dump some output, along with a description for how/when it should be used, covers several issues around this problem. I'm not claiming this would solve the specific issues discussed in the post. Just a potentially helpful tip for others out there.
Do you have an example of this in practice? I'm having a hard understanding this and have a very similar problem of the agent wanting to give a response on optional fields.
A (hopefully) clear and probably oversimplified example:
Query -> Person Lookup -> Result-> Structured Output `{ firstName: "", lastName: "" }`
When result doesn't have relevant information, structured output will basically always output a name, whether it found the correct person or not, because it wants to output something, even if the fields are optional. With this example, prompting can help turn the names into "Unknown", but the prompt usually ends up being excessive and/or time consuming to get correct and fix edge cases for. Weaker models might struggle more on details or relevance with this prompt-only approach.
`{ found: boolean, missingFields: [], missingReason: "", firstName, lastName }`
Including one or more of these additional text output properties has an almost magical affect sometimes, reducing the required prompting and hallucinated/incorrect outputs.
Re: Learnings from building AI agents
#70what I saw using 5-6 tools like this: - PR description is never useful they barely summarize the file changes - 90% of comments are wrong or irrelevant wether it's because it's missing context, missing tribal knowledge, missing code quality rules or wrongly interpret the code change - 5-10% of the time it actually spots something Not entirely sure it's worth the noise
the code reviews can't be effective because the LLM does not have the tribal knowledge and product context of the change. it's just reading the code at face value
Isn't it possible to feed that knowledge and context to it? Have it scan your product website and docs, code documentation, git history, etc?