I'm not sure it's even "ready for the server". I have a server without a monitor I'd like to VNC to. Unfortunately X doesn't start without a physical monitor plugged in because it uses it for autodetection. It's such a silly problem that you can even buy fake "monitor" dongles to plug in to trick X into thinking that there is one. I don't feel like spending money on a software problem so I followed the standard worka…
Using x11vnc to connect to a VNC server is like SSHing into localhost instead of just opening up a terminal. You're approaching the task the wrong way around.
Wifi not reconnecting sounds like your system has a graphical widget handling the network connections. No X = no graphical programs = no network connections.