Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta developer tools: Working at scale

engineering.fb.com

Post reply on HN
Print view View original

151–160 of 228 posts

Re: Meta developer tools: Working at scale

#151

Post by DannyBee »

kayson wrote: post #50

I know git is complex, and the UX is sometimes messy, but after really, really learning it (shoutout to the Github training folks), I've never had any problems that couldn't be solved. I understand the desire to simplify some things, and their log looks way better than gits, but I wish they'd contribute back instead of rolling their own entire VCS. The one thing that is super exciting to me is the stacked pull reques…

I know git makes you shove toothpicks under your fingernails in order to let you use the keyboard, but after really, really learning to do this, i've never found it a blocker for my daily work. I understand the desire to simplify things, but i wish they'd contribute back ways of making people more comfortable with the toothpicks rather than just removing them and starting from scratch.

I am not a fan of FB, but they tried - you can find them on the git mailing list where they got told they are doing it wrong for things like "scaling" or "productivity". Which is always ironic since basically nobody in open source generates or uses any real data about productivity, it's all just gut feelings about users.

Re: Meta developer tools: Working at scale

#152

Post by TechBro8615 »

spookthesunset wrote: post #132
jonathankoren wrote: post #114

Earlier quoted context omitted.

And now it's 2023, and they're inventing the wheel again.

Not a very robust analogy. There are tons of different wheel designs out there. Some are good for racing and suck for rain. Some are great in the snow but are noisy and inefficient on the highway. Etc... Maybe their needs are so special they need a wheel design that doesn't commercially exist?

Someone really needs to reinvent that analogy.

Re: Meta developer tools: Working at scale

#153

Post by rangledangle »

innagadadavida wrote: post #69
rangledangle wrote: post #42

Earlier quoted context omitted.

No company I've worked at after FB/Meta ships or works even at a non-eng level with the same velocity. I always attributed that to their internal tools, since most of the other companies seem to be using the same crap. Slack is straight up painful compared to their chat system. And don't get me started on the how good the task tool is compared to literally any other ticketing system out there. Everything is behind th…

Shipping faster should not be the only metric, the amount of SEVs should be given much higher weight. The attitude of most engineers is to show impact and if there is a SEV, it is better as it will show even more impact. This slows other people down and causes a gradual decline is actually shipping things that matter - quality over quantity.

Oh yea, I elaborated more in another comment. But the ease of finding documentation related to code, SEVs, anything wrong. Someone pushing something and breaking something else in my env, then me using the amazing tools like diff to see what was pushed recently that affects my realm, I can quickly track down, find, and often times alter to fix my problem with no more than a message to the original author in a comment on the new diff.

Early career this is pretty huge for growth. I was there until 2019 though, so not sure how it is now.

My point was that the cohesion of all these internal tools makes information discovery frictionless. cross-pollination of functional space in a business is like butter because everything is built by Meta and behind their intern tool. The other companies I've worked at just don't have this. They lack the Eng capacity to build it, unfortunately.

Re: Meta developer tools: Working at scale

#154

Post by hyuuu »

zug_zug wrote: post #19

I think Meta's tooling is inferior to industry standard. I actually took a survey while I was there, and that wasn't the majority opinion, but frankly I think most outsiders would absolutely agree. Things like Eden were a great idea, but tools had all sorts of issues they gloss over (a virtual file system can be really slow if you have a ton of small files), dev environments would randomly fail a lot, really the only…

as a former employee who has worked both in many other companies over a decade + starting a few startups, Meta tooling is one of the best if not the best, it's so good to a point where Saas companies came out of facebook simply by replicating their internal tooling, like Asana, Scuba, etc

Re: Meta developer tools: Working at scale

#155

Post by ahahahahah »

cletus wrote: post #129
ahahahahah wrote: post #116

Earlier quoted context omitted.

Everyone reading this should assume it's talking about FB around 2014 or so, I don't know how it could otherwise be so wrong. > FB's code search is just grep. It's literally called BigGrep. This hasn't been true for a long time, codesearch at FB is more complex than just grep and has some semantic understanding. Here's some discussion here when some of this infrastructure was open sourced: https://news.ycombinator.co…

As for Glean, you concede "some semantic understanding". All I know is as of 2 years ago I couldn't do "Find Usages" of a particular C++ method where I could Google cs 6+ years ago. I did spend way more time doing www than C++ though and Hack was specifically written to facilitate regex searches like being able to search for "SomeClassName::someFunctionName" to find usages as well as the other examples of prohibiting…

> First, there's more to FB builds than infra C++, most notably iOS and Android, which are all built locally

Again, ios and Android have had the remote artifact caching you mention as so important since many years ago. Android has had remote execution for years (I think even before the infra c++ you mention).

> But a devserver build required a full checkout

This is not true with sapling, which had been used extensively for years.

Re: Meta developer tools: Working at scale

#156

Post by thfuran »

likpok wrote: post #121
blitz_skull wrote: post #4

Sapling looks quite cool! I've used git extensively in my career and consider myself as having a slightly-more-advanced-than-typical understanding of how to use it just based on conversations with colleagues. However, one thing that's always been very limiting with git has been stack-based PR reviews, and as they mentioned amending deep commits. It's not impossible, but it makes it awkward enough that I usually avoid…

When we first switched from git to hg internally I was real cranky. Git was the clear industry winner, and hg was adding a whole bunch of churn for no good reason. Now, hg is amazing and when I leave I will be extremely sad to go back to git. The UX is well thought out, with commands mapping to operations (how do you undo a commit? hg uncommit vs git playing around with the reflog). Amending deep commits is pretty go…

The main point against hg is that the tooling is dying out as git becomes even more dominant. But I do quite like it.

Re: Meta developer tools: Working at scale

#157

Post by tm-guimaraes »

idkyall wrote: post #26
zug_zug wrote: post #19

I think Meta's tooling is inferior to industry standard. I actually took a survey while I was there, and that wasn't the majority opinion, but frankly I think most outsiders would absolutely agree. Things like Eden were a great idea, but tools had all sorts of issues they gloss over (a virtual file system can be really slow if you have a ton of small files), dev environments would randomly fail a lot, really the only…

Yeah, that's fair. I think another thing is that a lot of these internal tooling projects date back 5-10 years when open source alternatives may not have been viable yet. For example, my company uses an in-house written time series database, which probably made sense at the time because when the project started, Prometheus wasn't 1.0 yet. Now it's reaching growing pains as we've scaled, and at this point it's a bit o…

> For example, my company uses an in-house written time series database, which probably made sense at the time because when the project started, Prometheus wasn't 1.0 yet.

That's why big orgs are now opensourcing their tools. But it's not to clean up all "too internal" stuff from a tool.

Re: Meta developer tools: Working at scale

#158

Post by zug_zug »

optymizer wrote: post #60
zug_zug wrote: post #19

I think Meta's tooling is inferior to industry standard. I actually took a survey while I was there, and that wasn't the majority opinion, but frankly I think most outsiders would absolutely agree. Things like Eden were a great idea, but tools had all sorts of issues they gloss over (a virtual file system can be really slow if you have a ton of small files), dev environments would randomly fail a lot, really the only…

For what it's worth, as a Meta employee, I generally disagree with this comment. I mean, calling it 'inferior' is just laughable. It's way better than what open source projects have access to and it's way better than Amazon internal tooling (is that 'industry standard' enough?). The sapling workflow is better than git. I was skeptical when I joined, but between no branches and excellent stack and merge support, it's…

>> Amazon internal tooling (is that 'industry standard' enough?).

I'm not sure, maybe?

When I say industry-standard tools I mean "Best you can purchase." So I mean a UI as good as github, a platform as good as AWS (meta's doesn't hold a candle to AWS), full-text log searching as good as splunk, chat as fast/searchable as slack (workplace doesn't hold a candle), video chat as fast/clear/clean as Zoom (workplace doesn't hold a candle)

Some problems are harder at scale (feature toggles interactions just get harder with size), but some of them are just a mess of their own making (restarting taking 30m, butterfly rules having >20m+ delays, eden not being optimized to work with buck, which needs to read tons of files out of tons of directories)

Re: Meta developer tools: Working at scale

#159

Post by vlovich123 »

zug_zug wrote: post #158
optymizer wrote: post #60

Earlier quoted context omitted.

For what it's worth, as a Meta employee, I generally disagree with this comment. I mean, calling it 'inferior' is just laughable. It's way better than what open source projects have access to and it's way better than Amazon internal tooling (is that 'industry standard' enough?). The sapling workflow is better than git. I was skeptical when I joined, but between no branches and excellent stack and merge support, it's…

>> Amazon internal tooling (is that 'industry standard' enough?). I'm not sure, maybe? When I say industry-standard tools I mean "Best you can purchase." So I mean a UI as good as github, a platform as good as AWS (meta's doesn't hold a candle to AWS), full-text log searching as good as splunk, chat as fast/searchable as slack (workplace doesn't hold a candle), video chat as fast/clear/clean as Zoom (workplace doesn'…

GitHub's developer workflow is a joke compared with the ticketing, CI, and code review tooling at Meta. I found the tools at Meta made it a lot smoother and faster to write and review code. Also, their automated flaky test detection, suppression, & ticket issuance/resolution is no joke. One of my favorite features. Their feature flagging system was also on point. Also, Workplace is a fantastic replacement of Wiki that surfaces relevant & interesting content to you. Their chat tools are way better than Google Chat / Microsoft Teams (not sure how it stacks up against Slack since I'm not a huge user of them).

I agree the observability tools I interfaced within Meta was subpar a few years ago, but I think you're being ungenerous to take that and extend it to the coding tooling specifically.

Re: Meta developer tools: Working at scale

#160

Post by wilde »

optymizer wrote: post #60
zug_zug wrote: post #19

I think Meta's tooling is inferior to industry standard. I actually took a survey while I was there, and that wasn't the majority opinion, but frankly I think most outsiders would absolutely agree. Things like Eden were a great idea, but tools had all sorts of issues they gloss over (a virtual file system can be really slow if you have a ton of small files), dev environments would randomly fail a lot, really the only…

For what it's worth, as a Meta employee, I generally disagree with this comment. I mean, calling it 'inferior' is just laughable. It's way better than what open source projects have access to and it's way better than Amazon internal tooling (is that 'industry standard' enough?). The sapling workflow is better than git. I was skeptical when I joined, but between no branches and excellent stack and merge support, it's…

> Eden is fast. Crazy fast, in fact, if you look at the size of the monorepo.

You’re measuring the wrong thing. The user actions are slow and take several seconds to complete usually. No one gives a shit how big the repo is. They care how long their operation takes.

Post reply on HN