Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook's Top Open Data Problems

research.facebook.com

Post reply on HN
Print view View original

51–60 of 65 posts

Re: Facebook's Top Open Data Problems

#51

Post by bachback »

ransom1538 wrote: post #44

I had a really great time talking to the Facebook engineers during my interviews there. The main pattern I noticed was Harvard (i was applying in management). Even more so, the guys interviewing me were extremely talented and smart. What always weirded me out was... the problems they work on are not that difficult. Once you grasp sharding and operations, you are pretty much set. These guys are not the Manhattan proje…

true - Facebook scales linearly. If you're interested in really hard problems in distributed systems & blockchains let me know. Facebook should not own social data. the future will be end-to-end encrypted and de-centralised.

Re: Facebook's Top Open Data Problems

#52

Post by trsohmers »

IndianAstronaut wrote: post #46
EricBurnett wrote: post #6

I strongly dislike Facebook the product, and to lesser extent Facebook the company, but I'm continually impressed with Facebook's approach to engineering in the open. I find this an interesting dichotomy. Would I want to work there? I still don't think so, but my opinion on that front is getting less strong over time.

I was interviewing with them some time ago and just gave up half way. Those guys are absolute assholes and are hugely arrogant. Not a place I would want to work.

Assuming you are referring to the hardware group and it was after 2010 (when the current team formed), then I have no idea why you would think that. I interviewed there back in 2013, and was quite impressed by how open and nice they were. While not working at Facebook, I have continued to work within the Open Compute Project, and the FB engineers have been great in my experience.

Re: Facebook's Top Open Data Problems

#53

Post by beagle3 »

valarauca1 wrote: post #13
beagle3 wrote: post #5

Earlier quoted context omitted.

Thanks. So, how much does the 4PB of data go down to when it goes into hive? My guess would be something like 200TB. Is it zipped (or lz4/lzo/zopfli/lzma whatever)? Or is it just "distilled"?

99% of raw data is useless. Just as a rule. Most of it likely gets tossed into a program to determine if somebody actually needs to do anything, or if something is actually breaking. For example raw user interaction doesn't really grow. While the event is likely ~1kb or so of raw data, at the end your just incrementing a 64bit counter. This is a baseless exaggerated post but should shed some light on.

> 99% of raw data is useless. Just as a rule.

Well, as a non-facebook user, I think 99.9999% of facebook data is useless :) But facebook is in the habit of tracking everyone's surfing habits across the web (through "like" links), and I assume they do more than just "increment a counter with it", even if they don't keep every single detail.

Re: Facebook's Top Open Data Problems

#55

Post by tiler »

e12e wrote: post #50
ransom1538 wrote: post #44

I had a really great time talking to the Facebook engineers during my interviews there. The main pattern I noticed was Harvard (i was applying in management). Even more so, the guys interviewing me were extremely talented and smart. What always weirded me out was... the problems they work on are not that difficult. Once you grasp sharding and operations, you are pretty much set. These guys are not the Manhattan proje…

No idea how much you're trolling, but the single file: https://github.com/memcached/memcached/blob/master/memcached... is more like 4000 lines of C (by guesstimating the amount of comments etc).

Nice guess. There are 4486 lines of C when the set of lines that start with {/* , * , */} + the set of empty lines are excluded.

Re: Facebook's Top Open Data Problems

#56

Post by mFixman »

EricBurnett wrote: post #6

I strongly dislike Facebook the product, and to lesser extent Facebook the company, but I'm continually impressed with Facebook's approach to engineering in the open. I find this an interesting dichotomy. Would I want to work there? I still don't think so, but my opinion on that front is getting less strong over time.

Former Facebook intern here. Facebook the company is a lot more 'hack-y' in the right way of the word than what it looks like on the outside. The company and its products are extremely open, and the projects you do working there have extremely little management and corporate bs.

My experience was really similar to my Google internship, and probably even closer to a "cool startup". I know several people who worked on Google, Facebook, and X (with X another major Silicon Valley company) and say that the first two were a lot closer to each other than to X.

Re: Facebook's Top Open Data Problems

#57

Post by Kiro »

mFixman wrote: post #56
EricBurnett wrote: post #6

I strongly dislike Facebook the product, and to lesser extent Facebook the company, but I'm continually impressed with Facebook's approach to engineering in the open. I find this an interesting dichotomy. Would I want to work there? I still don't think so, but my opinion on that front is getting less strong over time.

Former Facebook intern here. Facebook the company is a lot more 'hack-y' in the right way of the word than what it looks like on the outside. The company and its products are extremely open, and the projects you do working there have extremely little management and corporate bs. My experience was really similar to my Google internship, and probably even closer to a "cool startup". I know several people who worked on…

Where do you work now?

Re: Facebook's Top Open Data Problems

#58

Post by mqsiuser »

e12e wrote: post #50
ransom1538 wrote: post #44

I had a really great time talking to the Facebook engineers during my interviews there. The main pattern I noticed was Harvard (i was applying in management). Even more so, the guys interviewing me were extremely talented and smart. What always weirded me out was... the problems they work on are not that difficult. Once you grasp sharding and operations, you are pretty much set. These guys are not the Manhattan proje…

No idea how much you're trolling, but the single file: https://github.com/memcached/memcached/blob/master/memcached... is more like 4000 lines of C (by guesstimating the amount of comments etc).

I liked his comment (don't shoot me for that), made me smile.

4000 isn't that much either & whats memchached? A hashmap?

He maybe means 300 lines of relevant code (+ ~3700 sugar)

Re: Facebook's Top Open Data Problems

#59

Post by rebootthesystem »

EricBurnett wrote: post #6

I strongly dislike Facebook the product, and to lesser extent Facebook the company, but I'm continually impressed with Facebook's approach to engineering in the open. I find this an interesting dichotomy. Would I want to work there? I still don't think so, but my opinion on that front is getting less strong over time.

Well, The Facebook company can feel rather juvenile to deal with. Some of their decisions in dealing with other businesses sometimes feel like they are being made by fifteen year old kids with no life or business experience.

In terms of the Facebook product one cannot deny the obvious: They touched THE nerve on the internet. World wide. Across languages and cultures.

I don't like the embodiment of the product at all. At the very least it has usability and privacy problems. Yet more people use it successfully than any other web app in the world. So, what do I know? What do experts know?

A similar thing could be said about CraigsList. It's 2014. Every time I use the site I cannot believe what I am looking at. Yet I and lots of other people keep using it. It works.

Facebook has a general lack of elegance (whatever that means). The kind that happens when a product is thrown together and evolved over time. Evolving anything over time means the output "naturally selects" (subverting the theory here) to the environment created by it's users.

They survive because they optimized for what is important to their users. Grandma couldn't care less about UI issues or searchability. She wants to see her grandchildren's pictures and videos. And for that it works very well for a huge percentage of the planet.

At some point it becomes almost impossible to break the mold and clean-up what might be less than ideal. Why would you? It works. Another "Innovator's Dilemma" [1] situation to a large extent.

[1] http://www.amazon.com/The-Innovators-Dilemma-Revolutionary-B...

Re: Facebook's Top Open Data Problems

#60

Post by JoeAltmaier »

rebootthesystem wrote: post #59
EricBurnett wrote: post #6

I strongly dislike Facebook the product, and to lesser extent Facebook the company, but I'm continually impressed with Facebook's approach to engineering in the open. I find this an interesting dichotomy. Would I want to work there? I still don't think so, but my opinion on that front is getting less strong over time.

Well, The Facebook company can feel rather juvenile to deal with. Some of their decisions in dealing with other businesses sometimes feel like they are being made by fifteen year old kids with no life or business experience. In terms of the Facebook product one cannot deny the obvious: They touched THE nerve on the internet. World wide. Across languages and cultures. I don't like the embodiment of the product at all.…

Glad somebody said that. Facebook is clumsy and weird. It looks like a legacy system, which I suppose it is. I don't even understand the basic premise. You have a 'wall'? Is that a thing anymore? Maybe its a feed now, or a timeline or something. Whatever; its not obvious from the front page. Not obvious what to do, or where to go, or how to friend people, or what will happen if you do that.
Post reply on HN