While I missed out on the CP/M train, I do remember GEM very well. I grew up in a printing company and spent a lot of time in the prepress parts of the company - around the layout and art folks.
There was one employee who brought his own desktop publishing setup. It was a souped up desktop PC and some other kit, but it had the latest MS-DOS, Windows 2.x, Windows 3.x, and GEM - plus a bunch of art and desktop publishing software (I believe Pagemaker).
It wasn't entirely clear at the time what the future was, so it was perfectly reasonable to have all this in one place. I remember at some point noting how similar GEM was to GEOS on my friend's Commodore 64, but how much snappier it felt and how much crisper it looked.
I remember Computer Chronicles being on the air at that time, but never connected the dots that what I was looking at on the computer had anything to do with the co-host. It was a bit dry of a show for a kid, but consistent with the day. But even then I remember Gary clearly being an expert, and I identified much more with him than with Stewart.
A few years later when my family finally bought a PC compatible the only offering for operating system and desktop environment came from Microsoft. They had simply annihilated everything else on the market. By then all the software I knew from the desktop publishing suite and my friends ran in either MS-DOS or Windows 3.1 -- or it ran on an entirely different system (C64, Macintosh, etc.). So we just sort of accepted it.
I remember at some point getting a modem, dialing into BBSs and coming across GEM on a warez board. I downloaded it, had some memories, then moved on. It never evolved the software ecosystem around it that Microsoft managed to create.