Puppet and Chef are great tools, but my first entry into AWS management was a little more attainable without having to learn anything new.
I simply set up the instance to automatically download the current codebase from our 'production' repo on every boot, automatically install requirements, run database migrations, etc. Then freeze that AMI as the production AMI.
If you migrate your databases off of the instance to RDS, don't use EBS, and manage user uploads and static assets to go straight to S3, then what you end up with a very easily load balanceable configuration.
To set up load balancing, just spin up two of those instances, attach them to an elastic load balancer, attach the load balancer to your elastic IP, then you ought to be more redundant and management free. (Note, this doesn't fix your security issues, though you could very easily bake a nightly apt-get run for security updates into that image).
After that, you want to look into Fabric (or language-specific alternatives if they exist) to allow for remote management of your machines for things like bouncing web services, pulling new code deploys, etc.
If you want to sidestep all this management altogether, I'd strongly suggest looking into something like dotcloud - http://dotcloud.com/ - which effectively does all this for you.