Facebook's Top Open Data Problems • Hacker News Board

Re: Facebook's Top Open Data Problems

#61

Post by tkinom » Wed, Nov 05, 2014, 4:43 PM UTC

nbm wrote: post #40

Cakez0r wrote: post #37

Earlier quoted context omitted.

Yes, that's the mechanism I'm wondering about. Let's say I have: Application -> TAO -> Cache -> Database I have a photo node (P1) and 125 comments nodes [C1, C2, ..., Cn] attached to P1 by the edges [(P1,C1), (P1,C2), ..., (P1,Cn)]. I'll ignore the fact that there can be different edge types for simplicity. Lets say my page size is 50 and I want to view 3 pages of comments for the photo from my application. My applic…

There is no memcache - there is just TAO, and it talks to the database. TAO is a read/write-through cache, so the only way the data changes in the database is through TAO. TAO contains the indices necessary to answer queries like that efficiently (although there may be other systems for doing similar but more specialized or slightly different queries), as well as a cache of the data. So, when you add a comment, TAO u…

Nice, BTW, TAO itself is not opensource? right. :-)

I love to try build a golang version of TAO as good mental exercise.

Anyone if something similar in golang or any other open source packages exist already?

Anyone else interested in such thing?

Re: Facebook's Top Open Data Problems

#62

Post by tkinom » Wed, Nov 05, 2014, 5:05 PM UTC

nbm wrote: post #40

Cakez0r wrote: post #37

Earlier quoted context omitted.

Yes, that's the mechanism I'm wondering about. Let's say I have: Application -> TAO -> Cache -> Database I have a photo node (P1) and 125 comments nodes [C1, C2, ..., Cn] attached to P1 by the edges [(P1,C1), (P1,C2), ..., (P1,Cn)]. I'll ignore the fact that there can be different edge types for simplicity. Lets say my page size is 50 and I want to view 3 pages of comments for the photo from my application. My applic…

There is no memcache - there is just TAO, and it talks to the database. TAO is a read/write-through cache, so the only way the data changes in the database is through TAO. TAO contains the indices necessary to answer queries like that efficiently (although there may be other systems for doing similar but more specialized or slightly different queries), as well as a cache of the data. So, when you add a comment, TAO u…

Are the TAO APIs asynchronous? Are there any document on the parameters of those APIs? :-)

Re: Facebook's Top Open Data Problems

#63

Post by mFixman » Wed, Nov 05, 2014, 7:02 PM UTC

Kiro wrote: post #57

mFixman wrote: post #56

Earlier quoted context omitted.

Former Facebook intern here. Facebook the company is a lot more 'hack-y' in the right way of the word than what it looks like on the outside. The company and its products are extremely open, and the projects you do working there have extremely little management and corporate bs. My experience was really similar to my Google internship, and probably even closer to a "cool startup". I know several people who worked on…

Where do you work now?

I'm currently finishing my degree while working in a smallish start-up in Buenos Aires.

Re: Facebook's Top Open Data Problems

#64

Post by e12e » Thu, Nov 06, 2014, 3:31 AM UTC

mqsiuser wrote: post #58

e12e wrote: post #50

Earlier quoted context omitted.

No idea how much you're trolling, but the single file: https://github.com/memcached/memcached/blob/master/memcached... is more like 4000 lines of C (by guesstimating the amount of comments etc).

I liked his comment (don't shoot me for that), made me smile. 4000 isn't that much either & whats memchached? A hashmap? He maybe means 300 lines of relevant code (+ ~3700 sugar)

Hehe. Well, this is just one file. I might agree that memcached is 300 lines of relevant pseudocode -- but then again, it's implemented in C, not pseudocode...

I agree that 4k lines isn't that much, but it's an order of magnitude off from 300. And again, that's just for that one single file.

Re: Facebook's Top Open Data Problems

#65

Post by Cakez0r » Fri, Nov 07, 2014, 12:16 PM UTC

tkinom wrote: post #62

nbm wrote: post #40

Earlier quoted context omitted.

There is no memcache - there is just TAO, and it talks to the database. TAO is a read/write-through cache, so the only way the data changes in the database is through TAO. TAO contains the indices necessary to answer queries like that efficiently (although there may be other systems for doing similar but more specialized or slightly different queries), as well as a cache of the data. So, when you add a comment, TAO u…

Are the TAO APIs asynchronous? Are there any document on the parameters of those APIs? :-)

There is a whitepaper about the details of TAO at https://www.cs.cmu.edu/~pavlo/courses/fall2013/static/papers...