Disclaimer: I am far from a security expert.
If your website is used by authenticated users i.e. requiring username password to login, then you could use their password to encrypt relevant data and decrypt it in runtime when the user logs in. However make sure you destroy the password and decrypted information from memory after user logs out or after a certain timeout, whichever is earlier.
Since now you are not storing keys on your server, attacker wont be able to decrypt sensitive information even if he gains administrative access to your server. Each user will be holding key in form of his password. The attacker will have to get password of each user.
Use https so that the attacker wont be able to sniff out password of the user.
Downside:
1. Forgot password functionality will be hard to implement
2. If users try to login to your application after attacker gains access to it, attacker will come to know of the password and use it to decrypt information pertaining to that user.
3. Even you won't be able to see the information since you won't have the decryption key.