Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Superintelligence Labs' first paper is about RAG

paddedinputs.substack.com

Post reply on HN
Print view View original

171–180 of 283 posts

Re: Meta Superintelligence Labs' first paper is about RAG

#171

Post by pityJuke »

ipsum2 wrote: post #17

This has nothing to do with superintelligence, it's just the people that were working on the paper prior to the re-org happened to publish after the name change. Though it is notable that contrary to many (on HN and Twitter) that Meta would stop publishing papers and be like other AI labs (e.g. OpenAI). They're continued their rapid pace of releasing papers AND open source models.

What model(s) have Meta released since the Lab re-org?

Also, that wasn't based on purely hearsay, Zuck explicitly said:

> We believe the benefits of superintelligence should be shared with the world as broadly as possible. That said, superintelligence will raise novel safety concerns. We'll need to be rigorous about mitigating these risks and careful about what we choose to open source. Still, we believe that building a free society requires that we aim to empower people as much as possible. [0]

[0]: https://www.meta.com/superintelligence/

Re: Meta Superintelligence Labs' first paper is about RAG

#172

Post by smokel »

bradleyjg wrote: post #165
smokel wrote: post #134

Earlier quoted context omitted.

> I certainly didn't judge them because they are just playing the game. Please do judge them for being parasitical. They might seem successful by certain measures, like the amount of money they make, but I for one simply dislike it when people only think about themselves. As a society, we should be more cautious about narcissism and similar behaviors. Also, in the long run, this kind of behaviour makes them an annoyi…

but I for one simply dislike it when people only think about themselves The key word there is only. Nothing in the post you suggested only. You have one vignette about one facet of this guy’s life. I really dislike the resurgence in Puritanism.

Please don't read too much into this single word. The comment above mentioned "nearly every ounce of energy they expended on research was strategic", and I was keeping that in mind while writing my remark.

Please read my sibling comment where I expand a bit on what I meant to say.

Re: Meta Superintelligence Labs' first paper is about RAG

#173

Post by bobxmax »

tsunamifury wrote: post #45
thereitgoes456 wrote: post #39

Earlier quoted context omitted.

The reportings at the time said that he was Mark’s 5th choice or similar. It is fairly clear he would prefer Ilya, Murati, Mark Chen, and perhaps others, but they said no, and Alex Wang was the first one to say yes.

Why in the world would he want Murati? She has absolutely no technical chops and was not functionally CTO of OpenAI.

What technical chops does Sam Altman have?

Re: Meta Superintelligence Labs' first paper is about RAG

#174

Post by sdeframond »

ekianjo wrote: post #136
ipsum2 wrote: post #17

This has nothing to do with superintelligence, it's just the people that were working on the paper prior to the re-org happened to publish after the name change. Though it is notable that contrary to many (on HN and Twitter) that Meta would stop publishing papers and be like other AI labs (e.g. OpenAI). They're continued their rapid pace of releasing papers AND open source models.

Open weights models, not open source. And even their weights are under a specific license not as permissive as apache 2.

I propose that from now on we call freewares "open binaries".

Re: Meta Superintelligence Labs' first paper is about RAG

#175

Post by meindnoch »

hamasho wrote: post #62
godelski wrote: post #15

It's kinda funny, Meta has long had some of the best in the field, but left them untapped. I really think if they just took a step back and stop being so metric focused and let their people freely explore then they'd be winning the AI race. But with this new team, I feel like meta mostly hired the people who are really good at gaming the system. The people that care more about the money than the research. A bit of th…

My theory is that as more people compete, the top candidates become those who are best at gaming the system rather than actually being the best. Someone has probably studied this. My only evidence is job applications for GAFAM and Tinder tho.

Goodhart's law

Re: Meta Superintelligence Labs' first paper is about RAG

#176

Post by impossiblefork »

aerhardt wrote: post #166
impossiblefork wrote: post #157

Earlier quoted context omitted.

We can't throw in infinite things in the context though. My impression is that GPT-5 gets confused, not quite right away, but after a couple of pages it has no idea. It doesn't take pages on pages before it forgets things.

I’m currently experimenting with prompts of ~300k tokens for a certain classification task and I think I might be able to make it work. GPT5 chokes but Gemini 2.5 Pro is showing promise. Jury’s still out and I might change my tune in a couple of weeks.

It should also be said, that what I say here is focused on things where these models have problems.

For example, I consider the model confused when it starts outputting stereotyped or cliche responses, and I intentionally go at problems that I know that the models have problems with (I already know they can program and do some maths, but I want to see what they can't do). But if you're using them for things they're made for, and which aren't confusing, such as people arguing with each other, then you are probably likely to succeed.

Prompts with lots of examples are reasonable and I know they can get very long.

Re: Meta Superintelligence Labs' first paper is about RAG

#177

Post by gdulli »

xvector wrote: post #103
gdulli wrote: post #101

Earlier quoted context omitted.

Not everyone has given in to the crutch.

That's why I still use an abacus.

The abacus skills are safely obsolete, the skills of general thinking and creativity must not become that. This couldn't be more specious.

Meme thinking like this, repeating something you've heard as reflex without regard to whether it fits a situation, is the exact kind of unoriginality we can't allow to become the default mode of thinking.

Re: Meta Superintelligence Labs' first paper is about RAG

#179

Post by mark_l_watson »

A great idea, bypassing as much conversion as possible between vector space and natural language tokens. Reminds me of a discussion of having AI’s “talk” to each other using vector space.

There was an interesting quote “plain old BM25 from 1994 outperforms vector search on recall” and super relevant to what I did yesterday. I am trying to use small local models more often and yesterday I wrote Common Lisp code that uses a large corpus of text and a user query or prompt to construct a fairly concise one-shot prompt with select context from the text corpus. This is RAG, and I used both BM25 and vector embeddings matching. I added the code and an example as a new chapter in my CL book (link directly to new material: https://leanpub.com/lovinglisp/read#leanpub-auto-autocontext...) yesterday afternoon. BM25 is fast. This is new code, and I will certainly be experimenting more with it, but as-is it is useful when working with small local LLMs.

Re: Meta Superintelligence Labs' first paper is about RAG

#180

Post by RataNova »

ipsum2 wrote: post #17

This has nothing to do with superintelligence, it's just the people that were working on the paper prior to the re-org happened to publish after the name change. Though it is notable that contrary to many (on HN and Twitter) that Meta would stop publishing papers and be like other AI labs (e.g. OpenAI). They're continued their rapid pace of releasing papers AND open source models.

Still, I think the optics matter... the fact that Meta's still putting out technical work (and open sourcing it) after the restructure says a lot about where they want to position themselves
Post reply on HN