"Just don't quit Photoshop when you're done using it, and don't turn the computer off, put it to sleep. If you have an SSD (like your iPhone), then swapping Photoshop back into main memory will also be much faster." My experience in OS X is that Photoshop has to be restarted very frequently or it just eats up more and more memory until it uses it all. Seems like bad memory management.
If you have photoshop running and using a lot of RAM it is likely that were you to run another application the RAM usage of photoshop would drop to accommodate the other program as the OS will simply drop some of photoshop's pages and let the other program use them.
There is not really any performance hit from doing this since any page that is not "dirty" (i.e has not been written to) which will be basically any pages containing program code does not need to be swapped back to disk.
If you are concerned you should find a way to measure the quantity of IO going on in the swap area since heavy swap IO will be the best indicator that you either need more memory or something is not as it should be.
For example I run a server that is often at 70% or so of RAM usage but has only swapped something like 10MB of data in the last month.