Everything you said is true and part of the problem. I do want to clarify some things though that people may not know.
There are several reasons the back-office literally can't tell you what the price will be. First, is that if you have insurance, they don't know. It is up to the payor (insurance company) what the final price will be. Even though there are negotiated rates, most procedures have up to 5 service levels. So depending on the service level, the price will be different. And the service level is based on what service was provided which by definition hasn't happened yet. The only way to actually do it would be to be more like a garage. It is $50 to do a level 2 office examination and then I'll quote you on what we find out then and you can choose to continue or not. Most people would be annoyed by this.
Additionally, there are different codes for things like new patient vs existing patient. Seems simple, right? Actually if you haven't been to your doctor in the last two year, you are a new patient regardless of how long they have been your doctor. The service levels mentioned above are why for years, you had do fill out a paper detailing your patient history every visit. This is called taking a family history and allows for higher service levels and higher reimbursement. This is finally changing now that the family history is stored in the EMRs.
So, we finally figure this all out. An office visit might be always fine but some procedures require a pre-authorization. The rules for this are unique to your employers specific health plan, not the generic Aetna (e.g.). They may require previous diagnosis codes in your chart to indicate you are a risk before agreeing to pay. For example, a 25 year old with blood in their stool may want a colon cancer screening but that is only approved for people 40+. There is back in forth between the office and the payor to figure this out.
Oh, we didn't mention that a lot of employers are now self funded. This means that up to a certain loss threshold X, the company pays the bills, not the payor. This is handled by a company called a TPA (third party administrator). Obviously there are deductibles and other things they control and keep track of as well. This new procedure may now put the employer over their stop loss for the year. The TPA now needs to submit this bill to the re-insurer. They may require even more or different documentation in order for them to process and pay this claim. This is why offices have such huge back office components because a lot of the bills just get bounced back. They have to figure out why and resubmit.
So why are the list prices we see on our bill so high when insurance pays so little but the doctor accepts it anyway? It is mostly a hedge against mistakes. Again, since each employer plan may pay out differently even for the same insurance (there can be carve outs), the doctors are incentivized to over price and let the TPA and payor figure it out. If they accidentally charged $150 for something that is reimbursed at $250, they lose money. It is better to charge $900 and just accept whatever they get. They now don't need to constantly update the price of thousands of CPT (current procedural terminology) codes and make sure they are always up to date.
And this is just part of it.... The only problem in healthcare is mis-aligned incentives. As long as they are allowed, nothing will change.
To your case, I would get a new doctor. Paying cash is not that rare anymore and they should know how to handle it. In addition, they seem to try to overcharge the payor and it got dropped to you in this case. Not sure that I would trust them moving forward.
This is the problem but what are the solutions? In my opinion, we need to change three things. Each one would be beneficial separately. 1) Move to national reference based pricing. Said more simply, no one can charge more than Medicare * 1.5 or 2. This gets rid of all the bull crap negotiations and the doctors can still do fine. 2) We need healthcare to move away from employment. This is a relic of WWII labor shortages and its time has come. Besides helping employees be more flexible, it also incentivizes payors to get people healthy. People are lazy and won't switch unless they have to. Today, if a payor pays for a wellness plan, that company is likely to switch to a competitor next year to save 1% and they now benefit from the healthier customer. 3) We need to split health insurance. Most insurance is against a tragedy or massive loss. A yearly physical is planned and not a loss but is managed together. We need health plans that are focused on wellness and prevention and are priced at $X/month. Health insurance moves to being used only for hospital visits, ambulance rides, getting cancer, heart attacks, etc. These now separate health insurance premiums can be tied to user behavior, just like car insurance and life insurance are today. I'm not saying this will make a perfect system but it is more transparent and a step in the right direction.
Last point, in the US part of the healthcare cost is our demand for extreme convenience. They put in Urgent Cares all over town. The thought was that a lot of people who use the expensive ER services would be better served with a cheaper option. The hope was that ER usage would drop. It did not, people now just go to the Urgent Care in addition. Sally has a cold and we are at the mall so let's just go triple check she isn't dying. People have the right take their kids to the doctor every day if they want but all actions have consequences. You also can't complain that healthcare is so expensive when you are such a high utilizer. The Urgent Care has to pay a dozen people to sit there all day and someone has to pay for that convenient access.