I'd be most worried about the legal liability, what happens if someone starts using your service to download or sell pirated movies or child porn? Whats your legal exposure?
Yeah this is my issue. I'm not too worried about it being a successful business (more like a small side project), but I don't have the resources to fight any legal claims.
The technical side is pretty cheap compared to the legal manpower you need to protect your end users.