Live data from Hacker News

The Implications of Facebook Indexing a Trillion Posts

techcrunch.com

11–20 of 33 posts

Re: The Implications of Facebook Indexing a Trillion Posts

#11

And all of this is brought to you by the friendly folks of PHP. Next time when you bash PHP because the argument order for functions is inconsistent, think about how some people overlook semantics and build great things with the tools they have at their disposal.

How are you so sure the search index functionality is written in PHP?

Re: The Implications of Facebook Indexing a Trillion Posts

#12

And all of this is brought to you by the friendly folks of PHP. Next time when you bash PHP because the argument order for functions is inconsistent, think about how some people overlook semantics and build great things with the tools they have at their disposal.

Not all of it. Some of it, maybe. And given that they've forked PHP maybe even less than some.

PHP sometimes doesn't get the credit it deserves for what can be accomplished with it, particularly modern PHP... but i'm not entire sure it deserves credit for this in any reasonable way.

Re: The Implications of Facebook Indexing a Trillion Posts

#13
The "Deactivate your account" button is still the only privacy setting you need to click. The implications are only for facebook - now they have more rope to play with - and something else to maintain - this does nothing for users, who expressly waive their right to privacy from the outset in setting up an account.

Re: The Implications of Facebook Indexing a Trillion Posts

#14

> Most obviously, the News Feed could learn to mimic our external dialogue, showing us posts with similar content to what we spread. Never talk about sports or babies? Facebook could eventually filter those out of your feed. Just shared your thoughts on Syria, celebrity gossip, or the police state? The algorithm could pull an audible and show you more about related news. I was pretty sure Facebook was already doing t…

Anecdotal of course, but I have a Facebook friend from political circles, and I see posts from him every day about the Dallas Cowboys. I ignore those and typically like or comment on political posts, but I still see Cowboys all the time.

Re: The Implications of Facebook Indexing a Trillion Posts

#15
post #6
post #4

Earlier quoted context omitted.

You could build an index per person, but rather than search 500 indexes, the index of each person would be of all their and their friends' posts, so you would only search one index.

Doesn't that make updating rather expensive - for each new post it needs to go into a lot of different search indexes? Off-topic: Wasn't there a start-up a while back that was allowing people to build their own personal search index for their social media content?

Sounds about right, but the benefit is that reading through every single person's personal index would be very fast. This can be useful in many ways, and it alone was probably the goal.

In fact it was probably already developed long before this feature was deployed to the general populace, and this new feature is just a token gesture of "giving back to users."

I doubt it'll see much use from end-users themselves.

Yes the search terms from users will be useful, but I don't see this as being a replacement for Google.

Re: The Implications of Facebook Indexing a Trillion Posts

#16

And all of this is brought to you by the friendly folks of PHP. Next time when you bash PHP because the argument order for functions is inconsistent, think about how some people overlook semantics and build great things with the tools they have at their disposal.

I would be willing to wager the shoes on my feet that this indexing system is not written in PHP.

Re: The Implications of Facebook Indexing a Trillion Posts

#18

The "Deactivate your account" button is still the only privacy setting you need to click. The implications are only for facebook - now they have more rope to play with - and something else to maintain - this does nothing for users, who expressly waive their right to privacy from the outset in setting up an account.

Deactivating is designed to leave your account intact in case you change your mind. In fact, some people use this as a super-logout, deactivating their account when not using it. If you want to truly delete your account, the best way to express that intention is the harder to find button to delete your account.

Re: The Implications of Facebook Indexing a Trillion Posts

#19
post #4

I'm pretty interested in how this is implemented, actually. Naively you could search a global index and then filter results down to your friends, but that filter seems impossibly slow. Alternatively you could maintain a separate index per person, but querying 500 indexes seems unreasonable. Maybe they partition the graph into cliques which they index, search and then fix up with a second pass to add and remove noncli…

You could build an index per person, but rather than search 500 indexes, the index of each person would be of all their and their friends' posts, so you would only search one index.

That's how Twitter does it. A new tweet is not immediately visible in all "subscribed" timelines. But eventually it is.

Re: The Implications of Facebook Indexing a Trillion Posts

#20

I'm pretty interested in how this is implemented, actually. Naively you could search a global index and then filter results down to your friends, but that filter seems impossibly slow. Alternatively you could maintain a separate index per person, but querying 500 indexes seems unreasonable. Maybe they partition the graph into cliques which they index, search and then fix up with a second pass to add and remove noncli…

There're two posts on Facebook's engineering blog that discuss this (they're from 2013 when they first started working on the feature):

https://www.facebook.com/notes/facebook-engineering/under-th...

https://www.facebook.com/notes/facebook-engineering/under-th...

Post reply on HN