Language is complicated, but I think I agree. Here are the most relevant bits I found just now in the Oxford English Dictionary:
master, n.1 and adj.
[...]
Origin: Of multiple origins. Partly a borrowing from Latin. Partly a borrowing from French. Etymons: Latin magistr-, magister; French maistre.
[...]
The classical Latin word has numerous Romance and Germanic derivatives, e.g. Italian maestro (compare maestro n.), Spanish maestro, Portuguese mestre (compare maistry n.); Old Frisian mastere, mestere (West Frisian master), Middle Dutch meester, meister, mester (Dutch meester), Old Saxon mēstar (Middle Low German mēster, mester, meyster, Low German meester), Old High German maistar, maister, meistar, meister (Middle High German meister, German Meister; compare Meistersinger n.), Old Icelandic meistari (Icelandic meistari), Norwegian (Nynorsk) meister, Old Swedish mæstare (Swedish mästare), Old Danish mester (Danish mester). See also mistress n.
Senses in all four branches are paralleled by uses in Latin, in branch A. III. initially reflecting use of Latin magister in the medieval universities.
With uses at branch A. IV., compare adjectival use in French in the sense ‘principal’ (applied to people from c1100 in Old French, to material objects and immaterial things from the 12th cent. in Anglo-Norman); compare also Italian maestro , used adjectivally in this sense from the 14th cent.
[..."Master-gate" was the closest I could find to master password or key...]
Compounds
C2. Applied in extended use to things, with the sense ‘main’, ‘principal’, ‘controlling’, ‘originating’. In such compounds master is generally construed as a noun, but in some cases the grammatical character of the compound is uncertain or fluctuating, and master may be considered to be functioning as an adjective.
a. Of material things. Esp. frequent in terms relating to building and the human body (in non-technical use), and later to machinery, electronics, etc.
master-gate n.
1340 Ayenbite (1866) 249 (MED) Þe mouþ..is þe mayster gate of þe castele of þe herte.
a1500 (▸?c1450) Merlin 422 He com to the maister yate of the paleys.
c1720 N. Dubois & G. Leoni tr. A. Palladio Architecture II. iii. 7 The chief Entry, or Master gate.
1998 Re: Sad Sight on Test Center in rec.games.computer.ultima.online (Usenet newsgroup) 8 Feb. I created a new character and walked through a Master Gate to do exactly what you do.