Earlier quoted context omitted.
That's pretty easily fixable if you train a LoRA or similar so that the model has a specific likeness in mind. (You can look at - and despair at - Civitai if you want proof.) It's harder to do at inference time without training, but I wouldn't assume it'll be impossible forever, especially with the existence of ControlNet.
This is a GAN, so you can just project the image of yourself into the latent space (which will give you a near-pixel-perfect reconstruction), fix the identity-relevant variables in the _z_, and edit it as necessary. (No workarounds like finetuning necessary. Just one of the many forgotten advantages of GANs.)
Is that how the latent space works though; Like if it's a 300-dim vector, is the face at locations 0-10?