Meta Superintelligence Labs' first paper is about RAG • Hacker News Board

Re: Meta Superintelligence Labs' first paper is about RAG

#81

Post by qingcharles » Sun, Oct 12, 2025, 4:59 AM UTC

xvector wrote: post #66

incompatible wrote: post #33

Earlier quoted context omitted.

I've heard of one study that said AI slows developers down, even when they think it's helping. https://www.infoworld.com/article/4061078/the-productivity-p...

AI may slow coding a bit but dramatically reduces cognitive load. The real value of AI isn't in helping coding. It's in having a human-like intelligence to automate processes. I can't get into details but my team is doing things that I couldn't dream of three years ago.

It does dramatically reduce cognitive load. I think that part is understated and lost to the headline of how it writes two thousand lines of code in 30 seconds.

Re: Meta Superintelligence Labs' first paper is about RAG

#82

Post by crystal_revenge » Sun, Oct 12, 2025, 5:01 AM UTC

godelski wrote: post #40

bobxmax wrote: post #28

Earlier quoted context omitted.

I thought Alex Wang was a very curious choice. There are so many foundational AI labs with interesting CEOs... I get that Wang is remarkable in his own right, but he basically just built MTurk and timed the bubble. Doesn't really scream CEO of AGI to me.

A lot of people also don't know that many of the well known papers are just variations on small time papers with a fuck ton more compute thrown at the problem. Probably the strongest feature that correlates to successful researcher is compute. Many have taken this to claim that the GPU poor can't contribute but that ignores so many other valid explanations... and we wonder why innovation has slowed... It's also weird…

> A lot of people also don't know that many of the well known papers are just variations on small time papers with a fuck ton more compute thrown at the problem.

I worked for a small research heavy AI startup for a bit and it was heart breaking how many people I would interact with in that general space with research they worked hard and passionately on only to have been beaten to the punch by a famous lab that could rush the paper out quicker and at a larger scale.

There were also more than a few instances of high-probability plagiarism. My team had a paper that had been existing for years basically re-written without citation by a major lab. After some complaining they added a footnote. But it doesn't really matter because no big lab is going to have to defend themselves publicly against some small startup, and their job at the big labs is to churn out papers.

Re: Meta Superintelligence Labs' first paper is about RAG

#83

Post by mirekrusin » Sun, Oct 12, 2025, 5:01 AM UTC

intelkishan wrote: post #63

puttycat wrote: post #42

Earlier quoted context omitted.

This was actually shown to not really work in practice.

I have seen this particular work example to work. You don't get the exact match but the closest one is indeed Queen.

Shouldn't this itself be a part of training?

Having set of "king - male + female = queen" like relations, including more complex phrases to align embeddings.

It seems like terse, lightweight, information dense way to address essence of knowldge.

Re: Meta Superintelligence Labs' first paper is about RAG

#84

Post by godelski » Sun, Oct 12, 2025, 5:02 AM UTC

vasco wrote: post #76

epwr wrote: post #73

Earlier quoted context omitted.

Could you elaborate or link something here? I think about this pretty frequently, so would love to read something!

Metric: time to run 100m Context: track athlete Does it cease to be a good metric? No. After this you can likely come up with many examples of target metrics which never turn bad.

So what is your argument, that it doesn't apply everywhere therefore it applies nowhere?

You're misunderstanding the root cause. Your example works as the the metric is well aligned. I'm sure you can also think of many examples where the metric is not well aligned and maximizing it becomes harmful. How do you think we ended up with clickbait titles? Why was everyone so focused on clicks? Let's think about engagement metrics. Is that what we really want to measure? Do we have no preference over users being happy vs users being angry or sad? Or are those things much harder to measure, if not impossible to, and thus we focus on our proxies instead? So what happens when someone doesn't realize it is a proxy and becomes hyper fixated on it? What happens if someone does realize it is a proxy but is rewarded via the metric so they don't really care?

Your example works in the simple case, but a lot of things look trivial when you only approach them from a first order approximation. You left out all the hard stuff. It's kinda like...

Edit: Looks like some people are bringing up metric limits that I couldn't come up with. Thanks!

Re: Meta Superintelligence Labs' first paper is about RAG

#85

Post by vasco » Sun, Oct 12, 2025, 5:04 AM UTC

naasking wrote: post #68

puttycat wrote: post #41

Seems very incremental and very far from the pompous 'superintelligence' goal.

A 30 fold improvement seems a tad more than incremental.

I can start brushing my teeth 30 times faster but it won't change my life. This is nice for RAG but it's a very localized improvement. And 30× sounds big but is just an order of magnitude improvement also.

Re: Meta Superintelligence Labs' first paper is about RAG

#86

Post by MR_Bulldops » Sun, Oct 12, 2025, 5:05 AM UTC

vasco wrote: post #76

epwr wrote: post #73

Earlier quoted context omitted.

Could you elaborate or link something here? I think about this pretty frequently, so would love to read something!

Metric: time to run 100m Context: track athlete Does it cease to be a good metric? No. After this you can likely come up with many examples of target metrics which never turn bad.

Do you have an example that doesn't involve an objective metric? Of course objective metrics won't turn bad. They're more measurements than metrics, really.

Re: Meta Superintelligence Labs' first paper is about RAG

#87

Post by btilly » Sun, Oct 12, 2025, 5:05 AM UTC

puttycat wrote: post #41

Seems very incremental and very far from the pompous 'superintelligence' goal.

If you can collapse "retrieve this complex chunk when it is needed" into a single token, what else can you put into a token?

"Send this through the math coprocessor." "Validate against the checklist." "Call out to an agent for X." "Recheck against input stream Y." And so on.

Retrieval augmentation is only one of many uses for this. If this winds up with better integration with agents, it is very possible that the whole is more than the sum of its parts.

Re: Meta Superintelligence Labs' first paper is about RAG

#88

Post by vasco » Sun, Oct 12, 2025, 5:06 AM UTC

godelski wrote: post #84

vasco wrote: post #76

Earlier quoted context omitted.

Metric: time to run 100m Context: track athlete Does it cease to be a good metric? No. After this you can likely come up with many examples of target metrics which never turn bad.

So what is your argument, that it doesn't apply everywhere therefore it applies nowhere? You're misunderstanding the root cause. Your example works as the the metric is well aligned. I'm sure you can also think of many examples where the metric is not well aligned and maximizing it becomes harmful. How do you think we ended up with clickbait titles? Why was everyone so focused on clicks? Let's think about engagement…

> So what is your argument, that it doesn't apply everywhere therefore it applies nowhere?

I never said that. Someone said the law collapses, someone asked for a link, I gave an example to prove it does break down in some cases at least, but many cases once you think more about it. I never said all cases.

If it works sometimes and not others, it's not a law. It's just an observation of something that can happen or not.

Re: Meta Superintelligence Labs' first paper is about RAG

#89

Post by alex-nt » Sun, Oct 12, 2025, 5:06 AM UTC

rhetocj23 wrote: post #53

crorella wrote: post #31

Earlier quoted context omitted.

Personal experience here in a FAANG, there has been a considerable increase in: 1. Teams exploring how to leverage LLMs for coding. 2. Teams/orgs that already standardized some of the processes to work with LLMs (MCP servers, standardized the creation of the agents.md files, etc) 3. Teams actively using it for coding new features, documenting code, increasing test coverage, using it for code reviews etc. Again, perso…

Im sure the MBA folks love stats like that - theres plenty that have infested big tech. I mean Pichai is an MBA+Mckinsey Alumni. Ready for the impending lay off fella?

There are places that offer Copilot to any team that wants it, and then behind the scenes they informed their managers that if the team (1+ persons) adopts it they will have to shed 10%+ human capacity (lose a person, move a person, fire a person) in the upcoming quarters next year.

Re: Meta Superintelligence Labs' first paper is about RAG

#90

Post by ccortes » Sun, Oct 12, 2025, 5:12 AM UTC

vasco wrote: post #76

epwr wrote: post #73

Earlier quoted context omitted.

Could you elaborate or link something here? I think about this pretty frequently, so would love to read something!

Metric: time to run 100m Context: track athlete Does it cease to be a good metric? No. After this you can likely come up with many examples of target metrics which never turn bad.

> Does it cease to be a good metric?

Yes if you run anything other than the 100m