I'm building a new service that allows users to subscribe to each other's feeds. In real time when publishers and consumers are both online (i.e. me and my friends) I use a message queue to take the published post and distribute it to my friends. However, where I'm struggling is how do I handle those friends that are not currently online? I can simply save new posts to the DB and pull them out when that user comes online, however I don't know the best way to retrieve an older post, check the friends its creator has and then send that message to all of them if they are online. If not don't send them anything. I can do it by storing more stuff in the DB but it becomes very difficult and time consuming to do all these table look ups just to figure out what message needs to go where.
So I'm wondering how does facebook or twitter handles thousands of friends posting tens/hundreds of messages a day, and how do they keep it all synchronized?
Thanks, Roy