Earlier quoted context omitted.
Reminds me of my time spent being a customer to a real estate MLS. When industries traditionally rented access to information asymmetry, even if those rents have since diminished, there's a lingering cultural momentum not to "show your hand" by playing nice with information (in addition to these not being organizations with the insight and incentives aligned to be attractive or conducive to IT competence).
Ive worked with MLS data before. It was terrible. I am currently working with ACH, DDF and I am astounded that our financial system even operates. Because if there is one thing you want in payment processing, it's wildly innacurate and inconsistent data that is lacking documentation.
The protocol used by MLS servers is kind of standardized... expect not really. There's no single way, for example, to accomplish a full download of an initial dataset. Some MLSes let you order by id with an offset, so you can paginate properly. With others you must use date ranges - but you're assuming that a create or update timestamp is actually meaningful, and surprise - they are not. Some let you run multiple concurrent connections so you can increase data throughput, while others only permit one slow connection that makes it takes 6+ hours to download their entire database.
Finally: good luck figuring out when a record is flat out deleted. The MLS may have a separate table where deleted records are supposed to go, but that table is always empty. The MLS nukes data rows with no way for clients to detect the change other than by downloading the entire database from scratch to find the gaps. This requires running hours of queue jobs PER MLS every 24 hours. It's so ridiculously inefficient.
Yeah... fuck MLS. End rant.