How should we evaluate memory for AI agents?

This is very interesting. Question: I am building an LLM-free memory system, does the gpt-4o-mini full evaluation requirement from the docs mean that systems using an LLM must standardize on gpt-4o-mini, or are LLM-free Add/Search systems ineligible? Thanks

Re: How should we evaluate memory for AI agents?

#5

Post by IreneAI » Wed, Aug 12, 2026, 2:57 AM UTC

mrsalty wrote: post #4

This is very interesting. Question: I am building an LLM-free memory system, does the gpt-4o-mini full evaluation requirement from the docs mean that systems using an LLM must standardize on gpt-4o-mini, or are LLM-free Add/Search systems ineligible? Thanks

> Sorry for the concern, and thank you for your support of AML. Your submission is eligible. > > According to the AML requirements, if an LLM is used, it must be *GPT-4o-mini*. If no LLM is used, this requirement does not apply.

Re: How should we evaluate memory for AI agents?

#6

Post by mrsalty » Wed, Aug 12, 2026, 6:14 AM UTC

IreneAI wrote: post #5

mrsalty wrote: post #4

This is very interesting. Question: I am building an LLM-free memory system, does the gpt-4o-mini full evaluation requirement from the docs mean that systems using an LLM must standardize on gpt-4o-mini, or are LLM-free Add/Search systems ineligible? Thanks

> Sorry for the concern, and thank you for your support of AML. Your submission is eligible. > > According to the AML requirements, if an LLM is used, it must be *GPT-4o-mini*. If no LLM is used, this requirement does not apply.

awesome, thanks