Earlier quoted context omitted.
... after the first visit...
Unless your site is on the HSTS list ( https://src.chromium.org/viewvc/chrome/trunk/src/net/http/tr... for Chrome, Firefox use the same list by verifying by connecting via HTTPS and checking for a long HSTS time.)
Which was basically my original point, which is that if you want your site to be generally accessible by just typing in the domain name, you still can't just turn off port 80...
Which I guess is why google.com itself is still reachable on port 80.