Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bento: Jupyter Notebooks at Meta

engineering.fb.com

Post reply on HN
Print view View original

121–130 of 134 posts

Re: Bento: Jupyter Notebooks at Meta

#121

Post by crabbone »

sangnoir wrote: post #105
crabbone wrote: post #12

Earlier quoted context omitted.

A friend of mine is doing his PHD while being an intern at Meta. He does not share your excitement... at all. To summarize his complaints: a framework written a long while ago with design flaws that were cast in stone, that requires exorbitant effort to accomplish simple things (under the pretense of global integration that usually isn't needed, but even if was needed, would still not work).

How long has he been interning? Is it long enough for him to have learned how long the timescale big-tech roadmaps operate on? If he wants a feature, he better write it himself (if his PR doesn't conflict with an upcoming rewrite, coming "soon"), or lobby to get it slotted for the second quarter of 2026.

He started right about the time COVID started, so... about four years now, I think. I'm not sure if those were contiguous though.

I'm not sure what your idea about PRs and features has to do with the above... he's not there to work on the internal infra framework. He's there for ML stuff. Unfortunately, the road to the later goes through the former, but he's not really a kind of programmer who'd deal with Facebook's infrastructure and plumbing.

The point is, it's inconvenient. Is it inconvenient because Facebook works on a five-year plan basis or whatever other reason they have for it doesn't really matter. It's just not good.

I also have no problems admitting that all big companies (two in total, one being Google) I worked for so far had bad internal tools. I don't imagine Facebook is anything special in this respect. I just don't feel like it's necessary to justify it in any way. It's just a fact of life: large companies have a tendency to produce bad internal tools (but small often have none whatsoever!) It's a water is wet kind of thing...

Re: Bento: Jupyter Notebooks at Meta

#122

Post by crabbone »

almostgotcaught wrote: post #88
crabbone wrote: post #12

Earlier quoted context omitted.

A friend of mine is doing his PHD while being an intern at Meta. He does not share your excitement... at all. To summarize his complaints: a framework written a long while ago with design flaws that were cast in stone, that requires exorbitant effort to accomplish simple things (under the pretense of global integration that usually isn't needed, but even if was needed, would still not work).

> A friend of mine is doing his PHD while being an intern at Meta I interned thrice as phd student at FB. your friend isn't entirely wrong but also just doesn't have enough experience to judge. all enormous companies are like this. FB is far and away better than almost all such companies (probably only with the exception of Google/Netflix).

I don't see why saying that Facebook's tools are bad should be invalidated by saying that Google's or others' tools are bad too. Google being bad doesn't vindicate or improve Facebook tools. There's no need for perspective: if it doesn't work well for what's it designed to do, then that's all there is to it.

Re: Bento: Jupyter Notebooks at Meta

#123

Post by extr »

KaiserPro wrote: post #109
hnav wrote: post #77

Earlier quoted context omitted.

Doesn't sound like your type of company tbh, the flipside is that a "serious" company will often have broken bs too except now nobody is going to look at your contribution/fix.

Pfft. "your type of company" sod off. Meta is only like this because its got a massive advertising revenue stream. the sheer amount of engineering time wasted because we don't document stuff is astounding. For example, how many message queue systems do we have? how many half arsed message queues have been created because they didn't know about FOQS?

Yes lmao, the number of times I would start off on some nominally useful task only to find out 3 weeks later that there is actually already a solution to that created by team XYZ that nobody in my reporting chain has ever heard of…(3 weeks was optimistic case, I remember my team member getting like 2 months in to some new data pipeline before finding out some tables already existed that did what he needed…)

Re: Bento: Jupyter Notebooks at Meta

#124

Post by ec109685 »

quicklime wrote: post #96
master_crab wrote: post #91

Earlier quoted context omitted.

If git is blocking you, you are using it wrong. Lotta instances of people treating it as an artifact repository. Use it correctly with a branching strategy that works for your use case and it's bulletproof. Plenty of other customers with the same magnitude problems as Meta are using Git perfectly fine.

Who are the others with the same magnitude as Google and Meta’s monorepos?

Microsoft has all of windows in a single repository.

Re: Bento: Jupyter Notebooks at Meta

#125

Post by ec109685 »

KaiserPro wrote: post #110
master_crab wrote: post #91

Earlier quoted context omitted.

If git is blocking you, you are using it wrong. Lotta instances of people treating it as an artifact repository. Use it correctly with a branching strategy that works for your use case and it's bulletproof. Plenty of other customers with the same magnitude problems as Meta are using Git perfectly fine.

> Plenty of other customers with the same magnitude problems as Meta are using Git perfectly fine. I mean there aren't. there are perhaps three places that have the same scale problem. A monorepo for a place with about 50k developers, that has been operating at that scale for 5 years. The current checkout if not sparse would be >80gigs The commit rate is > 20 a second. no amount of branching strategy is going to help…

Microsoft has made git work for massive repositories through a lot of blood sweat and tears: https://github.blog/open-source/git/the-story-of-scalar/

Re: Bento: Jupyter Notebooks at Meta

#126

Post by quantadev »

chthonicdaemon wrote: post #101
quantadev wrote: post #76

The original "Block Editor" (that Jupyter modeled itself after) is the one that's now called "Quanta", and has been around for decades in various forms and incantations: https://github.com/Clay-Ferguson/quantizr I'm thinking that Jupyter might still not be "Tree Based" but that would be a heck of a leap in capability if they "fix" that.

I always thought Jupyter was based on other notebook-style interfaces like Mathematica or Maple.

I meant the "block editor" aspect, like how individual chunks of text and images can be independently selected and moved around or even shared with their own URL.

I've long believed some system like that could and should some day replace even HTML and the web, and that it'll only happen if the Semantic Web ever takes off in a big way where chunks of stuff are "typed" (like a Type-Safe Web). Even Tim Berners Lee has been dreaming of this for decades, but the world is still stuck in HTML-land for the foreseeable future.

Re: Bento: Jupyter Notebooks at Meta

#127

Post by apwell23 »

3eb7988a1663 wrote: post #80
scottyah wrote: post #54

Earlier quoted context omitted.

I'm not sure anyone has access to the real data, but I've had a suspicion that Netflix is able to remain a lot more profitable due to their superior tech. Cloud hosting and streaming (not to mention labor) can get very expensive, and I think while it's easier to set up nowadays (in comparison to when they started) a lot of the other companies are burning cash to try to keep up. HBO Max (just Max now?) has always had…

Surely that has more to do with having a fleet of edge nodes that mirror content close to consumers? There are only so many ways to ship video bytes across the internet. Best way to save money is to move fewer bytes.

Yep its simply a money problem not some sort of tech genius by netflix. Max would be as good as netflix if they were willing to spend money on edge.

Re: Bento: Jupyter Notebooks at Meta

#128

Post by zhengyi13 »

lacker wrote: post #67
zhengyi13 wrote: post #57

Earlier quoted context omitted.

Am I completely off-base/confused thinking that the GFE originally started life (like back under csilver) as a fork of boa[0]? [0]: http://www.boa.org/

I thought it was GWS that originally started as a fork of boa.

That's it, yes, thank you.

Re: Bento: Jupyter Notebooks at Meta

#129

Post by sangnoir »

crabbone wrote: post #121
sangnoir wrote: post #105

Earlier quoted context omitted.

How long has he been interning? Is it long enough for him to have learned how long the timescale big-tech roadmaps operate on? If he wants a feature, he better write it himself (if his PR doesn't conflict with an upcoming rewrite, coming "soon"), or lobby to get it slotted for the second quarter of 2026.

He started right about the time COVID started, so... about four years now, I think. I'm not sure if those were contiguous though. I'm not sure what your idea about PRs and features has to do with the above... he's not there to work on the internal infra framework. He's there for ML stuff. Unfortunately, the road to the later goes through the former, but he's not really a kind of programmer who'd deal with Facebook's…

> I'm not sure what your idea about PRs and features has to do with the above... he's not there to work on the internal infra framework.

My idea is if he's not making the monorepo codebase changes himself, he's going to have to wait for an awfully long time for any non-trivial improvements he'd like because the responsible teams have different priorities sketched out for next calendar year. It's a function of organization size, unless you have the support of someone very high up on the org chart, ICs can't unilaterally adjust another teams priorities.

Re: Bento: Jupyter Notebooks at Meta

#130

Post by almostgotcaught »

crabbone wrote: post #122
almostgotcaught wrote: post #88

Earlier quoted context omitted.

> A friend of mine is doing his PHD while being an intern at Meta I interned thrice as phd student at FB. your friend isn't entirely wrong but also just doesn't have enough experience to judge. all enormous companies are like this. FB is far and away better than almost all such companies (probably only with the exception of Google/Netflix).

I don't see why saying that Facebook's tools are bad should be invalidated by saying that Google's or others' tools are bad too. Google being bad doesn't vindicate or improve Facebook tools. There's no need for perspective: if it doesn't work well for what's it designed to do, then that's all there is to it.

> Google's or others' tools are bad too

lol bruh read my response again - FB's and Google's and Amazon's tool are lightyears ahead of #ARBITRARY_F100_COMPANY. you haven't a clue what "bad" means if you've never worked in a place that has > 1000 engineers.

Post reply on HN