Some suggested reasons, to add to the others cited:
0. You're obviously speaking mostly about startups that primarily interface with moving money or physical goods. Regulations and infrastructure in these areas are not insignificant operations. US startups working outside these areas (e.g. Facebook, Zynga, Stack Overflow) don't stop at US borders.
- Each market has its own customs, language, and tone. Marketing into a new country cannot be an afterthought, or it will not be successful.
- Language is a big deal. If you translate your website into German, you create the expectation that your customers will be able to email you questions in German and get help in German. You don't have resources for this unless it is a necessary step on the path to success. Even Stripe's $20mm isn't enough for this: remember that they are competing against very big, rich, public companies like eBay and Amazon.
- There is competition at home, in a very large, familiar, homogeneous market. If you are based in the USA and lose the American market, you will likely fail. (In fact, someone could become a competitor by starting the US-only version of your service to take advantage of your distraction.) So you need to win there first. Startups by definition have not won any markets yet.
- Non-US markets in industrialized countries are in general not all that big individually compared to the US market. Germany is like 20% the size of the US market; the UK about 12%, and Australia 10%. So it's really hard to get any major additional scale in one country, even after you jump through all the hoops to launch there and service their customers.
- US states are big economically. A startup trying to scale can probably get more mileage out of a big marketing push in a single state than by launching internationally. One sample choice: launch in the Netherlands (~15 hour flight from CA, different language, customs, regulations) or make a big push in Florida (~5 hour flight from CA, you're likely already in compliance with all their laws, your website is already in their language, your support staff speaks their language, etc.). Florida's economy is about the same size as the Netherlands'. Texas (~3h flight from CA, 4 metro areas over 2m people) has an economy and population roughly the size of Australia.
Startups need every advantage they can get to not take the default startup path (failure). The bottom line is that launching outside the US is not a barrier to ultimate success (your eBay example), but does introduce risks. So it's not often going to be a priority.