Just played with Gemini Ultra for like 10-15 mins, and right off the bat, it made mistakes I've never seen GPT-4 do. To give you an example, I asked Gemini Ultra how to set up a real-time system for a TikTok-like feed that matches card difficulty with user ability. It correctly mentioned "Item Response Theory (IRT)", which was a good start. But when I followed up asking how to implement a real-time IRT system, it sud…
I was starting to get OK results with 'Pro', but I had to use special prompting tricks.
Tried 'Advanced' (Ultra), seems only marginally better so far.