Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Superintelligence Labs' first paper is about RAG

paddedinputs.substack.com

Post reply on HN
Print view View original

191–200 of 283 posts

Re: Meta Superintelligence Labs' first paper is about RAG

#191

Post by bwfan123 »

kcexn wrote: post #145
t_serpico wrote: post #105

Earlier quoted context omitted.

But there is no way to know who is truly the 'best'. The people who position and market themselves to be viewed as the best are the only ones who even have a chance to be viewed as such. So if you're a great researcher but don't project yourself that way, no one will ever know you're a great researcher (except for the other great researchers who aren't really invested in communicating how great you are). The system s…

The problem is that the best research is not a competitive process but a collaborative one. Positioning research output as a race or a competition is already problematic.

right. Also, the idea that there is a "best" researcher is already problematic. You could have 10 great people in a team, and it would be hard to rank them. Rating people in order of performance in a team is contradictory to the idea of building a great team. ie, you could have 10 people all rated 10 which is really the goal when building a team.

Re: Meta Superintelligence Labs' first paper is about RAG

#192

Post by iisan7 »

contrarian1234 wrote: post #100
godelski wrote: post #15

It's kinda funny, Meta has long had some of the best in the field, but left them untapped. I really think if they just took a step back and stop being so metric focused and let their people freely explore then they'd be winning the AI race. But with this new team, I feel like meta mostly hired the people who are really good at gaming the system. The people that care more about the money than the research. A bit of th…

> Labs used to hire researchers and give them a lot of free reign. I can't think of it ever really paying off. Bell Labs is the best example. Amazing research that was unrelated to the core business off the parent company. Microsoft Research is another great one. Lots of interesting research that .. got MS some nerd points? But has materialized into very very few actual products and revenue streams. Moving AI researc…

It paid off for PARC, iirc the laser printer justified lots of other things that Xerox didn't profit from but turned out to be incredibly important.

Re: Meta Superintelligence Labs' first paper is about RAG

#193

Post by cindyllm »

gopher_space wrote: post #190
contrarian1234 wrote: post #100

Earlier quoted context omitted.

> Labs used to hire researchers and give them a lot of free reign. I can't think of it ever really paying off. Bell Labs is the best example. Amazing research that was unrelated to the core business off the parent company. Microsoft Research is another great one. Lots of interesting research that .. got MS some nerd points? But has materialized into very very few actual products and revenue streams. Moving AI researc…

The problem here is management expecting researchers to dump out actionable insights like a chicken laying eggs. Researchers exist so that you can rifle through their notes and steal ideas.

[dead]

Re: Meta Superintelligence Labs' first paper is about RAG

#194

Post by CaptainOfCoit »

drexlspivey wrote: post #148
ekianjo wrote: post #136

Earlier quoted context omitted.

Open weights models, not open source. And even their weights are under a specific license not as permissive as apache 2.

Does an “open source” model the way you describe it exist or is it a mythical creature?

It does, but does it matter? Even if every software released in 2025 was proprietary, doesn't make their published binaries "open source" because no other software could be classified as "open source".

We name things based on what they are, not based on the lack of other things.

Re: Meta Superintelligence Labs' first paper is about RAG

#195

Post by idiotsecant »

smokel wrote: post #134
crystal_revenge wrote: post #80

Earlier quoted context omitted.

I've spent most of my career working, chatting and hanging out with what might be best described as "passionate weirdos" in various quantitative areas of research. I say "weirdos" because they're people driven by an obsession with a topic, but don't always fit the mold by having the ideal combination of background, credentials and personality to land them on a big tech company research team. The other day I was spend…

> I certainly didn't judge them because they are just playing the game. Please do judge them for being parasitical. They might seem successful by certain measures, like the amount of money they make, but I for one simply dislike it when people only think about themselves. As a society, we should be more cautious about narcissism and similar behaviors. Also, in the long run, this kind of behaviour makes them an annoyi…

This take is simply wrong in a way that I would normally just sigh and move on, but it's such a privileged HN typical pov that I feel like I need to address it. If a plumber did plumbing specifically because someone needed it and he would be paid, would you call them a narcissist? If a gardener built a garden how their customer wanted would you call them a narcissist? Most of the world doesn't get to float around in a sea of VC money doing whatever feels good. They find a need, address it, and get to live another day. Productively addressing what other people need and making money from it isn't narcissism, it's productivity.

Re: Meta Superintelligence Labs' first paper is about RAG

#196

Post by Workaccount2 »

fishmicrowaver wrote: post #23
Workaccount2 wrote: post #18

Earlier quoted context omitted.

I think at this point it's software devs looking for the value unlock. Non-software devs are actually making functional programs for themselves for the first time ever. The value is crazy.

Sure but I'm the real world do you think businesses are going to deploy piles of code into production generated this way? No, non technical people will continue to whip up MS PowerApps. AI generated code has no value to many businesses.

You don't need production level code to make your life easier.

You're missing the forest for the trees. Most people can't even make a block diagram, but they can explain what they have and what they want to do with it.

Re: Meta Superintelligence Labs' first paper is about RAG

#197

Post by xvector »

gdulli wrote: post #177
xvector wrote: post #103

Earlier quoted context omitted.

That's why I still use an abacus.

The abacus skills are safely obsolete, the skills of general thinking and creativity must not become that. This couldn't be more specious. Meme thinking like this, repeating something you've heard as reflex without regard to whether it fits a situation, is the exact kind of unoriginality we can't allow to become the default mode of thinking.

I am not the one being unoriginal here. You are thinking that AI will obsolete critical thinking, so there's no point developing with it.

However, in your moral crusade against using AI you are missing the big picture. No one is making you code with AI. But there are many things that you can only build if you use AI as a component.

The ability to plug a human-like decisionmaker into anything, anywhere massively expands what we can build. There are applications and use cases that you cannot even conceptualize without having the ability to plug AI in. This does not impacting critical thinking whatsoever.

Be original. Put your engineer hat on and think on what this new tool lets you build, that you couldn't beforehand.

Re: Meta Superintelligence Labs' first paper is about RAG

#198

Post by lkey »

idiotsecant wrote: post #195
smokel wrote: post #134

Earlier quoted context omitted.

> I certainly didn't judge them because they are just playing the game. Please do judge them for being parasitical. They might seem successful by certain measures, like the amount of money they make, but I for one simply dislike it when people only think about themselves. As a society, we should be more cautious about narcissism and similar behaviors. Also, in the long run, this kind of behaviour makes them an annoyi…

This take is simply wrong in a way that I would normally just sigh and move on, but it's such a privileged HN typical pov that I feel like I need to address it. If a plumber did plumbing specifically because someone needed it and he would be paid, would you call them a narcissist? If a gardener built a garden how their customer wanted would you call them a narcissist? Most of the world doesn't get to float around in…

You are comparing a skilled trade that commands ~100k annual compensation to positions that have recently commanded 100 million dollars in compensation upon signing, no immediate productivity required, as this talent denial is considered strategic.

You consider the person who expects eventual ethical behavior from people that have 'won' capitalism (never have to labour again) to be privileged.

Re: Meta Superintelligence Labs' first paper is about RAG

#199

Post by Palmik »

The observation about the "block-diagonal patterns" in RAG isn't new and has been exploited / explored before:

- https://arxiv.org/abs/2410.07590 (literally titled "Block-Attention for Efficient RAG")

- https://arxiv.org/abs/2409.15355v3

- https://arxiv.org/abs/2212.10947

The REFRAG paper does not cite any of these.

Re: Meta Superintelligence Labs' first paper is about RAG

#200

Post by ipsum2 »

pityJuke wrote: post #171
ipsum2 wrote: post #17

This has nothing to do with superintelligence, it's just the people that were working on the paper prior to the re-org happened to publish after the name change. Though it is notable that contrary to many (on HN and Twitter) that Meta would stop publishing papers and be like other AI labs (e.g. OpenAI). They're continued their rapid pace of releasing papers AND open source models.

What model(s) have Meta released since the Lab re-org? Also, that wasn't based on purely hearsay, Zuck explicitly said: > We believe the benefits of superintelligence should be shared with the world as broadly as possible. That said, superintelligence will raise novel safety concerns. We'll need to be rigorous about mitigating these risks and careful about what we choose to open source. Still, we believe that buildin…

That has always been the policy. To answer your question, Meta has released ~100 models since the Superintelligence Lab reorg.

https://huggingface.co/facebook/models

The most interesting ones to me are:

- CWM (Code world model), an LLM for coding https://github.com/facebookresearch/cwm

- DINOv3, A vision encoder https://ai.meta.com/dinov3/

- MAPAnything, a 3d reconstruction model https://huggingface.co/facebook/map-anything

- VJEPA v2, Self-supervised video pre-training model https://github.com/facebookresearch/vjepa2

Post reply on HN