Something missing is the confidence that comes from being perceived that way as well. You're a high performer at a high prestige/competitive school - you're going to feel more like you can do everything the other successful people from that school can do compared to someone from a no-pedigree state school who wouldn't have access to those role models or tracks to see what's possible and be more likely to just go with the flow and accept a safe job offer.
Even if we assumed that there was nothing special about the capabilities of people who finish Ivy League educations, that's still a powerful effect. Add in that they are either competitive enough or have well connected enough families to get into said school, and it's a multiplying effect on top.