Here, this issue is that of availability. Given that it is sometimes difficult to gauge the desirability of a prospective partner, one metric is how eager they seem to be. If the other replies quickly, this is an indication that they are quite eager, which suggests that they have few encounters, and that therefore few have evaluated this person as attractive. On the contrary, a bit of a lapse before a response suggests that the other feels a bit of indifference toward you, and that therefore the person is likely to have more encounters, and therefore is considered more attractive.
You can be forgiven for wishing for more simplicity, given that relationships in most media are depicted rather one-dimensionally, as the result of unambiguous and mutual attraction. This is not how it tends to work in the real world.