Earlier quoted context omitted.
There's a whole village in North Wales that's Prisoner themed
The other way round, it predates the show and was used as the set.
But yes, Portmeirion had a rich history before McGoohan decided that it was perfect for his idea. Noël Coward wrote Blithe Spirit while staying in the village. George Bernard Shaw, H.G. Wells, Gregory Peck and Ingrid Bergman were all visitors. Beatles Paul McCartney, George Harrison and their manager, Brian Epstein, were also regulars in the village.