Meters are longer than yards, by about 8%. So 500yards is loosely about the same distance as 400 meters. But 500yards divides into ten 50-yards
laps, or 20
lengths of the pool. With the dive and the finish, that is 19 turns. At each turn they push off the wall and for a few seconds move much faster than when swimming in open water and a greater percentage of time underwater (which is faster). And the swimmers center of gravity doesn't get as close to the wall during a turn, effectively shortening the distance actually swam on every lap ending in a turn. But a 400-meter race in a 25-meter pool (roughly the same distance as 500 yards) has only 8 laps or 16-lengths. It has only 15 turns, meaning four fewer accelerations off the wall and less time underwater. All of these effects change based on the stroke, speed and even size of the swimmers. So there is no good direct comparison between yard and meter pools.
And then an olympic pool is 50 meters long, meaning far fewer turns for a given distance. So "long course" times are generally slower than short course even at the same distance.
(Underwater is so much faster that swimming has rules about how far you can travel underwater during each length.)