More recently I had a WiMAX plan with KDDI. WiMAX worked great and it also came with nationwide wifi access. While this worked in cafes, even at walking speeds leaving it set to automatically connect on my phone would cause my phone not to be connected half the time as it would be continuously joining and leaving wifi networks or occasionally get stuck on a login screen. Again I learned to leave it off.
The free networks from convenience stores and JR work in a pinch but you have to stand still and remember not to set it to automatically connect because the login is not persistent your phone will happily connect which effectively disconnects you from the internet.
Also note that allowable wifi transmission power is lower in Japan than many countries. Between that and congestion you’re lucky to connect 20m from the glass front of store with a router in it, let alone in any other direction.