I traced 3,177 API calls to see what 4 AI coding tools put in the context window
1–5 of 5 posts
Re: I traced 3,177 API calls to see what 4 AI coding tools put in the context window
#2Maybe I'm missing something, but I'm confused why you compared gemini 2.5 against opus 4.6, and gpt 5.3-codex ? I assume you used sonnet 4.5 because 4.6 was released in the last 2 days, but opus 4.6 and 5.3-codex were released 2 weeks ago, so gemini 3 was definitely around then.
Re: I traced 3,177 API calls to see what 4 AI coding tools put in the context window
#3Context lens is a great name!
Re: I traced 3,177 API calls to see what 4 AI coding tools put in the context window
#4Very interesting analysis and great write up. Gemini the semi.
Regarding Opus’s use of the git history: I run Opus and Codex in parallel a lot, or going back and forth with each other. For interesting problems I’ll give them both the same prompt at the same moment and see what they each come up with. My guess is that if you hadn’t told Opus that “it worked fine recently” it wouldn’t have gone first to git like that. Both Opus 4.6 and Codex 5.3 are incredibly good at reading code and diagnosing bugs.
Re: I traced 3,177 API calls to see what 4 AI coding tools put in the context window
#5Maybe I'm missing something, but I'm confused why you compared gemini 2.5 against opus 4.6, and gpt 5.3-codex ? I assume you used sonnet 4.5 because 4.6 was released in the last 2 days, but opus 4.6 and 5.3-codex were released 2 weeks ago, so gemini 3 was definitely around then.
It's a fair question. I've had problems with Gemini 3 due to rate limiting, and I've been working on this for a while now. I'm planning Gemini 3 for a follow up.