Earlier quoted context omitted.
"Starting to leave" is not the same as "leave" or "left". When I start to leave my house in the morning, I have not yet gone through the door, but I'm on my way. I walk towards the door to leave the house. I'm leaving the house as I pass through the door. I look back and can see that I've left the house.
But why are you starting to leave your house? Likely because you're ready to leave no? To me that's where it was implying that models are ready to leave that benchmark behind, and starting the process is coming up with a new benchmark. Anyways, if it's just semantic misunderstanding, I have no issue. That means the fact is, models still can't draw a pelican riding a bike? With maybe the exception of Fable 5?
Re: Karpathy’s Pelican
#461In this case, the preparation for "starting to leave" this benchmark would be getting a new benchmarks ready, because this one appears it will soon be obsolete.