> Is it simply a case of there not being enough competition between internet service providers?
This is a big part of it (2/3) of the US have 2 or fewer options for broadband.
> UPS and FedEx are free to charge a different price when delivering a package from Amazon.com compared to Walmart.com.
This is the wrong analogy. A better one is probably a gated community (which Wikipedia tells me is rare in Australia, unfortunately, so it may not be a good one for you).
The cast of this little play is as follows:
Netflix (played by amazon).
L3, Akamai: Played by UPS and FedEx
Comcast: played by the community association
You live in the gated community. You pay the association that runs the gated community for upkeep of the roads within the gate. Part of this includes a guard who lets delivery vehicles in.
Delivery vehicles used to be relatively low volume, but Amazon becomes popular, and all of the sudden UPS trucks are coming in a lot; as in to the point where the association needs to hire a second guard to let them in. The association refuses to do it unless UPS pays a monthly fee.
Around this time, the community association starts its own online store which residents can by from without any shipping delays.
Meanwhile, FedEx (which enters the community via a different gate) is having no trouble with deliveries. Amazon notices this and starts using FedEx in addition to UPS to solve the problem. Unfortunately the same pattern as with UPS happens.
People start complaining to Amazon and UPS, which results in a blog article from UPS talking about how the Community Association wants them to pay extra for more gates into the community; this makes no sense for 2 reasons:
1) The CA is there to serve the residents, and the residents are the ones who ordered from Amazon
2) Traditionally the CA pays for any infrastructure improvements within the gate, primarily due to #1
So the CA which is supposed to be serving the interests of those living within it is now abusing its status as gatekeeper to charge Amazon for the privilege of delivering packages to it.
This finally gets resolved with Amazon, under protest, paying to put a shipping warehouse within the gate, which is something they had earlier offered to do without charge.