Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta developer tools: Working at scale

engineering.fb.com

Post reply on HN
Print view View original

171–180 of 228 posts

Re: Meta developer tools: Working at scale

#171

Post by jarjoura »

vlovich123 wrote: post #159
zug_zug wrote: post #158

Earlier quoted context omitted.

>> Amazon internal tooling (is that 'industry standard' enough?). I'm not sure, maybe? When I say industry-standard tools I mean "Best you can purchase." So I mean a UI as good as github, a platform as good as AWS (meta's doesn't hold a candle to AWS), full-text log searching as good as splunk, chat as fast/searchable as slack (workplace doesn't hold a candle), video chat as fast/clear/clean as Zoom (workplace doesn'…

GitHub's developer workflow is a joke compared with the ticketing, CI, and code review tooling at Meta. I found the tools at Meta made it a lot smoother and faster to write and review code. Also, their automated flaky test detection, suppression, & ticket issuance/resolution is no joke. One of my favorite features. Their feature flagging system was also on point. Also, Workplace is a fantastic replacement of Wiki tha…

Phabricator w/ "arc & jf" were my bae. Wish they would fully open source all of it or expose a public version of it to compete against GitHub.

Re: Meta developer tools: Working at scale

#172

Post by nappy-doo »

optymizer wrote: post #60
zug_zug wrote: post #19

I think Meta's tooling is inferior to industry standard. I actually took a survey while I was there, and that wasn't the majority opinion, but frankly I think most outsiders would absolutely agree. Things like Eden were a great idea, but tools had all sorts of issues they gloss over (a virtual file system can be really slow if you have a ton of small files), dev environments would randomly fail a lot, really the only…

For what it's worth, as a Meta employee, I generally disagree with this comment. I mean, calling it 'inferior' is just laughable. It's way better than what open source projects have access to and it's way better than Amazon internal tooling (is that 'industry standard' enough?). The sapling workflow is better than git. I was skeptical when I joined, but between no branches and excellent stack and merge support, it's…

I've worked at Google and Meta (née FB). They are inferior.

Re: Meta developer tools: Working at scale

#173

Post by billjings »

Solvency wrote: post #124
MikeTheRocker wrote: post #71

Earlier quoted context omitted.

Having also written Android code at Meta, I 100% agree.

What's the short version of why Meta did this, wrapping so much up in this internal nonstandard format?

Often, performance.

When I was at Meta, if you could measure it, you could ship it and mark that as a win in your performance review. So a lot of projects got shipped off of A/B tested metric wins.

Re: Meta developer tools: Working at scale

#174

Post by ahahahahah »

cletus wrote: post #129
ahahahahah wrote: post #116

Earlier quoted context omitted.

Everyone reading this should assume it's talking about FB around 2014 or so, I don't know how it could otherwise be so wrong. > FB's code search is just grep. It's literally called BigGrep. This hasn't been true for a long time, codesearch at FB is more complex than just grep and has some semantic understanding. Here's some discussion here when some of this infrastructure was open sourced: https://news.ycombinator.co…

As for Glean, you concede "some semantic understanding". All I know is as of 2 years ago I couldn't do "Find Usages" of a particular C++ method where I could Google cs 6+ years ago. I did spend way more time doing www than C++ though and Hack was specifically written to facilitate regex searches like being able to search for "SomeClassName::someFunctionName" to find usages as well as the other examples of prohibiting…

> Typically, things are built on a devserver (although this was transitioning to on-demand VMs, for which the Google equivalent was CitC)

I skipped over that parenthetical in first read, but my God, it's not even wrong. I mean citc is excellent and all, but describing those as equivalent just makes me wonder if you held some non technical role and are just playing telephone from people who better understand things. Like I'm just imagining a conversation where you're like, how does Facebook handle this specific thing that's handled by a X at Google (or the reverse)? Getting an answer on one side that involves citc and the other that involves on demand and deciding that that specific problem represents the entirety of those two ecosystems.

I guess one of the difficulties I have here is that a lot of what you've described across a bunch of google-facebook comparison is just like so factually wrong that i can't ignore the problems with a direct reading of what you are saying. At a deeper level, I think I'd totally accept google's CITC and facebook's on-demand as like philosophically similar in that solutions to seemingly unrelated things fit into the structure enabled by those systems. Again though, I think crediting you with trying to discuss things in those terms would be too generous given that that equivalence is no more true of on-demand VMs than it is of devservers.

Re: Meta developer tools: Working at scale

#175

Post by o1y32 »

jonathankoren wrote: post #36

I stopped reading when I reached that they made their own CVS. This is a solved problem, and even if it isn’t, there’s entire communities dedicated to it. There’s literally no reason why Facebook needs to be in the business of reinventing the wheel from scratch beyond some dude trying to show “impact”. It’s a distraction from core business problems.

That's a very uninformed opinion. Nobody could say that unless they are capable of actually coming up with a solution that can handle version control at Meta's scale.

Re: Meta developer tools: Working at scale

#176

Post by School-Cotton »

castlecrasher2 wrote: post #51
rangledangle wrote: post #42

Earlier quoted context omitted.

No company I've worked at after FB/Meta ships or works even at a non-eng level with the same velocity. I always attributed that to their internal tools, since most of the other companies seem to be using the same crap. Slack is straight up painful compared to their chat system. And don't get me started on the how good the task tool is compared to literally any other ticketing system out there. Everything is behind th…

>Slack is straight up painful compared to their chat system. As someone who vastly prefers Slack over all its competitors, I'm very interested in hearing how Meta's compares. What makes it better than Slack for you?

Meta uses workchat which is just a reskinned Facebook Messenger. It’s been a while since I’ve used it but I remember it being a lot less glitchy than slack on iOS (desktop is another story).

Re: Meta developer tools: Working at scale

#177

Post by no_wizard »

rangledangle wrote: post #42
zug_zug wrote: post #19

I think Meta's tooling is inferior to industry standard. I actually took a survey while I was there, and that wasn't the majority opinion, but frankly I think most outsiders would absolutely agree. Things like Eden were a great idea, but tools had all sorts of issues they gloss over (a virtual file system can be really slow if you have a ton of small files), dev environments would randomly fail a lot, really the only…

No company I've worked at after FB/Meta ships or works even at a non-eng level with the same velocity. I always attributed that to their internal tools, since most of the other companies seem to be using the same crap. Slack is straight up painful compared to their chat system. And don't get me started on the how good the task tool is compared to literally any other ticketing system out there. Everything is behind th…

Its not the tools, its the culture. Meta cut red tape in its early years especially. I know there is more "process" now, but they empowered teams and individuals to just make decisions and move on and the ecosystem around everything supports this.

They'd be just as fast with Slack or whatever, for the most part.

EDIT: as someone posted below:

> The open-ness of code, visibility, diffs. It was perfection. Something broke in my env suddenly? OH, I just checked recently pushed diffs that affect my realm. Hey there it is, security pushed something weird. I'll just revert the part that affects me and tag them. No meeting, maybe a SEV for visibility and review, maybe not. Easy peasy.

This is the kind of empowerment developers are given and expected to handle (both the explicitly ability to revert previously committed code and the implicit responsibility that the teams code that was overridden must deal with it once they're notified rather than push back)

Re: Meta developer tools: Working at scale

#178

Post by jonathankoren »

sangnoir wrote: post #120
jonathankoren wrote: post #76

Earlier quoted context omitted.

[flagged]

For a moment, let's assume they do abandon the monorepo. What solution would you recommend for managing code dependencies and coordinating releases between thousands of teams (at a modest 5 repos per team) - git tags?

This coordinating releases across teams is not a unique a problem. In fact, every large software organization solves this problem. They don’t usually do it in a assbackwards way due to institutional blindness.

Re: Meta developer tools: Working at scale

#179

Post by jonathankoren »

spookthesunset wrote: post #130
jonathankoren wrote: post #113

Earlier quoted context omitted.

You might be surprised to learn that they're solving the wrong problem.

I don't think you fully understand how big their codebase is and how many different teams are working on it at any given time. There are very fundamental differences between a mono-repo and a bunch of repos for each "service" or whatever. Lots of tradeoffs. I've worked both and I can see the reasons for huge monorepos. They make a lot of things that were previously hard much simpler... The tradeoff is your tooling ne…

Sit down. I worked at FB.

Re: Meta developer tools: Working at scale

#180

Post by HumanOstrich »

c_crank wrote: post #30
ink_13 wrote: post #9

Earlier quoted context omitted.

Probably not, if only for reasons of inertia. Git plus third-party review tools (like GitHub) is more than "good enough" for most purposes. I used to work at FB, and while sapling is quite nice to use in practice, without the internal version of Phabricator to do code review (and, in all likelihood, mononoke), I don't think I'd pick it up again.

GitHub's gotten so much worse since Microsoft bought it. The drop in quality compared to when I used it in school is remarkable.

Can you provide some specific examples?
Post reply on HN