Does someone know how these types of projects come to fruition in a big company like Facebook? Are people working on them full time (with no other workload)? Do engineers build them on the weekend? How do they get 'funded'?
I think the key is that these projects actually provide value - they make things faster, more reliable, more scalable - whether that's the code's execution or the people writing the code (or debugging issues, or whatever).
They generally aren't solutions seeking problems - they are responses to problems that exist.
Engineers generally don't build things like this on the weekend - unless they like to structure their time like that, I guess. It may or may not be a full-time job, but the job whatever it is isn't some search for abstract perfection, it is again to solve real problems encountered by others in the company. Often it is a part-time component built as part of trying to solve some more direct goal - like fighting spam, or serving bits, or whatever.
Often it is something the engineers just do - it makes sense to break things up into libraries, or services, or whatever, and they do that, and then that library or service is usable/useful elsewhere, and that's it. Other times they may suggest and motivate it as a goal-in-itself in a team goal setting situation.
I doubt that's a particularly useful answer, but maybe with further questions I can make it more useful to you?