Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bento: Jupyter Notebooks at Meta

engineering.fb.com

Post reply on HN
Print view View original

91–100 of 134 posts

Re: Bento: Jupyter Notebooks at Meta

#91

Post by master_crab »

kridsdale3 wrote: post #56
tqi wrote: post #28

Earlier quoted context omitted.

Facebook actually doesn't use git, they use mercurial ( https://graphite.dev/blog/why-facebook-doesnt-use-git ). That decision is also illustrative of why they end up forking most things - Facebook's usage patterns at the far extreme end for almost any tool, and things thats are non-issues with fewer engineers or a smaller codebase become complete blockers.

Yes when I used to talk about this to interviewees, I described that every tool people commonly use is somewhere on the Big-O curves for scaling. Most of the time we don't really care if a tool is O(n) or O(10 n) or whatever. At Meta, N tends to be hundreds of billions to hundreds of trillions. So your algorithm REALLY matters. And git has a Big-O that is worse than Mercurial, so we had to switch.

If git is blocking you, you are using it wrong. Lotta instances of people treating it as an artifact repository. Use it correctly with a branching strategy that works for your use case and it's bulletproof.

Plenty of other customers with the same magnitude problems as Meta are using Git perfectly fine.

Re: Bento: Jupyter Notebooks at Meta

#92

Post by extr »

tru3_power wrote: post #87
extr wrote: post #58

Earlier quoted context omitted.

People probably think you’re exaggerating but it’s true. Sometimes when I would get blocked the suggestion was to “read the source code” or “submit a fix” on some far flung internal project. Huge fucking waste of time and effort, completely unserious.

No matter what, tools will be broken. Having access to the source and being able to land a diff to fix the issue is awesome imo.

That’s how open source already works by default. The difference is if an OSS tool is broken my boss doesn’t imply landing a fix is my responsibility on top of my regular job duties.

Re: Bento: Jupyter Notebooks at Meta

#93

Post by deadmutex »

steventhedev wrote: post #90
kridsdale3 wrote: post #56

Earlier quoted context omitted.

Yes when I used to talk about this to interviewees, I described that every tool people commonly use is somewhere on the Big-O curves for scaling. Most of the time we don't really care if a tool is O(n) or O(10 n) or whatever. At Meta, N tends to be hundreds of billions to hundreds of trillions. So your algorithm REALLY matters. And git has a Big-O that is worse than Mercurial, so we had to switch.

I'm gonna disagree with you there. The difference was with stat patterns, and the person at Facebook who ran the tests had something wrong with the disk setup that was causing it to run slowly. They ignored multiple responses that reproduced very different results. Nail in the coffin on this was a benchmark GitHub ran two years ago that got the results that FB should have: git status within seconds. Facebook didn't u…

This sounds plausible, but would love a source

Re: Bento: Jupyter Notebooks at Meta

#95

Post by aprilthird2021 »

Random_BSD_Geek wrote: post #4
web3aj wrote: post #2

The internal tools at Meta are incredible tbh. There’s an ecosystem of well-designed internal tools that talk to each other. That was my favorite part of working there.

Polar opposite of my experience. To achieve the technical equivalent of changing a lightbulb, spend the entire day wrangling a dozen tools which are broken in different ways, maintained by teams that no longer exist or have completely rolled over, only to arrive at the finish line and discover we don't use those lightbulbs anymore. Move things and break fast.

But you're both talking about different things. The tools are both often left in disuse, lacking documentation, etc. But they also have a really tight integration with each other that allows for unparalleled visibility and ability over enormous systems with many moving parts.

Re: Bento: Jupyter Notebooks at Meta

#96

Post by quicklime »

master_crab wrote: post #91
kridsdale3 wrote: post #56

Earlier quoted context omitted.

Yes when I used to talk about this to interviewees, I described that every tool people commonly use is somewhere on the Big-O curves for scaling. Most of the time we don't really care if a tool is O(n) or O(10 n) or whatever. At Meta, N tends to be hundreds of billions to hundreds of trillions. So your algorithm REALLY matters. And git has a Big-O that is worse than Mercurial, so we had to switch.

If git is blocking you, you are using it wrong. Lotta instances of people treating it as an artifact repository. Use it correctly with a branching strategy that works for your use case and it's bulletproof. Plenty of other customers with the same magnitude problems as Meta are using Git perfectly fine.

Who are the others with the same magnitude as Google and Meta’s monorepos?

Re: Bento: Jupyter Notebooks at Meta

#97

Post by ec109685 »

ElonChrist wrote: post #89
Random_BSD_Geek wrote: post #4

Earlier quoted context omitted.

Polar opposite of my experience. To achieve the technical equivalent of changing a lightbulb, spend the entire day wrangling a dozen tools which are broken in different ways, maintained by teams that no longer exist or have completely rolled over, only to arrive at the finish line and discover we don't use those lightbulbs anymore. Move things and break fast.

[dead]

Large checkouts is a solved problem now https://github.com/facebook/sapling/blob/main/eden/fs/docs/O...

Re: Bento: Jupyter Notebooks at Meta

#98

Post by ahupp »

ipsum2 wrote: post #36
JohnMakin wrote: post #23

Earlier quoted context omitted.

One of the crazier things a L4 meta colleague of mine told me, that I still don’t believe entirely, is that meta pretty much has their own fork of everything , even tools like git. is this true?

Yep. Zeus is a fork of Zookeeper, Hack is a fork of PHP, etc. It's usually needed to make it work with the internal environment. The few things that don't have forks are usually the open source projects like React or PyTorch, but even those have some custom features added to make it work with FB internals.

nit: HHVM was a completely new implementation of a runtime for a PHP-like language, it wasn't a fork of Zend.

Re: Bento: Jupyter Notebooks at Meta

#99

Post by disgruntledphd2 »

quicklime wrote: post #96
master_crab wrote: post #91

Earlier quoted context omitted.

If git is blocking you, you are using it wrong. Lotta instances of people treating it as an artifact repository. Use it correctly with a branching strategy that works for your use case and it's bulletproof. Plenty of other customers with the same magnitude problems as Meta are using Git perfectly fine.

Who are the others with the same magnitude as Google and Meta’s monorepos?

Particularly in 2014, when the git thing happened.

Re: Bento: Jupyter Notebooks at Meta

#100

Post by majormajor »

extr wrote: post #92
tru3_power wrote: post #87

Earlier quoted context omitted.

No matter what, tools will be broken. Having access to the source and being able to land a diff to fix the issue is awesome imo.

That’s how open source already works by default. The difference is if an OSS tool is broken my boss doesn’t imply landing a fix is my responsibility on top of my regular job duties.

Working around it is somehow is. A huge part of my work has been plumbing and hacking around limitations in mediocre-at-best OSS tools.

Lots of nonserious companies that take those issues as enough of a reason to move slowly.

Many fewer serious ones where bad tooling is expected to be fixed, smoothed over, or replaced entirely in the interest of future dev time.

Post reply on HN