Heroku does this - they have a nice console interface for pulling down the logs from your deployed app. Granted, it is not quite the same as Gmail, but it's always a start.
Maybe it's a business idea: email for geeks. I'd use it. :)
A request to cloud providers: Give us access to the logs, please?
11–15 of 15 posts
Re: A request to cloud providers: Give us access to the logs, please?
#12You can get at some of that information for gmail. See http://mail.google.com/support/bin/answer.py?ctx=gmail&a... for details.
Re: A request to cloud providers: Give us access to the logs, please?
#13Almost 10 years ago I worked for a digital imaging company called ACD Systems and we had a major product launch of ACDSee. We were using Akamai as a CDN and we wanted to know the stats about who was downloading.
Think about this for a second.. Akamai is massive with 10,000+ global servers handling massive amounts of traffic. It might sound simple to fetch one users logs but how do you make this simple for the user. Akamai also had a proprietary log format. To their credit we had a couple conference calls and we worked with their engineers to find a solution. They were a great company.
You don't see logs being offered because it is a major pain in the ass to compile these logs from many servers and reduce them for a specific user.
Re: A request to cloud providers: Give us access to the logs, please?
#14Re: A request to cloud providers: Give us access to the logs, please?
#15"This is not hard to do." That's a big assumption. I've worked with systems that wouldn't have been able to provide this without a LOT of work and upgrades, and they weren't anywhere near the size if Google and Amazon.
For a company at their size, they probably already have those internal logs in place (for debugging, security, etc). Would be nice if they opened it up to the en d users.
Even ignoring other commenters' valid concerns about sensitive data stored in logs, this can be a tough thing to scale.
Say the logs live on disk on a filer somewhere. Need to do a security investigation? OK, dig up the file the logs are in for that particular user and grep around. Lots of disk seeks, but pretty fast.
Scaling that to support production-scale queries is a significant undertaking. (Oh, you can only do 100 seeks per second, need on average 100 seeks per query, and desire to service a low demand of 100 qps? Not gunna happen!)