Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Superintelligence Labs' first paper is about RAG

paddedinputs.substack.com

Post reply on HN
Print view View original

201–210 of 283 posts

Re: Meta Superintelligence Labs' first paper is about RAG

#201

Post by hippo22 »

ekianjo wrote: post #136
ipsum2 wrote: post #17

This has nothing to do with superintelligence, it's just the people that were working on the paper prior to the re-org happened to publish after the name change. Though it is notable that contrary to many (on HN and Twitter) that Meta would stop publishing papers and be like other AI labs (e.g. OpenAI). They're continued their rapid pace of releasing papers AND open source models.

Open weights models, not open source. And even their weights are under a specific license not as permissive as apache 2.

I'm not a lawyer, but I believe that the weights aren't subject to copyright. So, you can use them outside of Meta's license agreement provided you get them from somewhere else.

Re: Meta Superintelligence Labs' first paper is about RAG

#202

Post by jeremyjh »

carom wrote: post #170
HPsquared wrote: post #162

Earlier quoted context omitted.

This is the right terminology. Model weights are literally compiled binary data; they are the output of an algorithm run on a bunch of source data. That training dataset is the "source" of the model. Training data (or the scripts used to generate it) is human-readable and modifiable, like source code. Binary weights are not.

Just to note though, source copyright extends to its compiled form. There is probably an analogue there for model weights.

Tell me about the companies that own the copyrights to their training data.

Re: Meta Superintelligence Labs' first paper is about RAG

#203

Post by godelski »

yellowcake0 wrote: post #150
godelski wrote: post #123

Earlier quoted context omitted.

Yes but it doesn't generalize very well. Even on simple features like gender. If you go look at embeddings you'll find that man and woman are neighbors, just as king and queen are[0]. This is a better explanation for the result as you're just taking very small steps in the latent space. Here, play around[1] mother - parent + man = woman father - parent + woman = man father - parent + man = woman mother - parent + wom…

so addition is not associative?

I think you're missing the point

Re: Meta Superintelligence Labs' first paper is about RAG

#205

Post by smeeger »

asim wrote: post #111

This was inevitable. You can't keep training LLMs and expect that's the answer to the evolution of AI. Yes it'll happen and we'll keep creating new more refined and bigger models but it's like DNA or something like the cortex of the brain. After that you need these systems that essentially "live" for years digesting information and develop a more refined way to process, store and retrieve the information. Compression…

exactly. the real focus internally is working on new architectures. there is no other possibility.

Re: Meta Superintelligence Labs' first paper is about RAG

#206

Post by tsunamifury »

bobxmax wrote: post #173
tsunamifury wrote: post #45

Earlier quoted context omitted.

Why in the world would he want Murati? She has absolutely no technical chops and was not functionally CTO of OpenAI.

What technical chops does Sam Altman have?

[flagged]

Re: Meta Superintelligence Labs' first paper is about RAG

#207

Post by tsunamifury »

hn_throwaway_99 wrote: post #47
tsunamifury wrote: post #45

Earlier quoted context omitted.

Why in the world would he want Murati? She has absolutely no technical chops and was not functionally CTO of OpenAI.

> was not functionally CTO of OpenAI. Why do you say that?

Her history was entirely non technical up until openAI.

Re: Meta Superintelligence Labs' first paper is about RAG

#208

Post by tsunamifury »

typpilol wrote: post #126
tsunamifury wrote: post #44

Earlier quoted context omitted.

Alexandr Wang is not interesting and a few steps short of a fraud that Mark had to bail out because he was so co invested. Shareholders should be livid if they knew a single thing about what was going on.

Tell me more

Scale promised cutting-edge data pipelines and model-training infra but mostly sold outsourced labeling with a tech veneer. Great margins, weak moat — classic Valley overclaim, not outright fraud.

Re: Meta Superintelligence Labs' first paper is about RAG

#209

Post by SknCode »

I am not sure if I understand things correctly.

I came to believe the LLMs work with token embeddings. Is then the REFRAG only "something" in front of the LLM, and the decoder is the RL policy which expands only some token chunk embeddings into token embeddings feedable to LLM? Or the REFRAG needs you to 'tune' the LLM to be able to work with both token embeddings and token chunk embeddings?

Re: Meta Superintelligence Labs' first paper is about RAG

#210

Post by throw_this_one »

gdulli wrote: post #177
xvector wrote: post #103

Earlier quoted context omitted.

That's why I still use an abacus.

The abacus skills are safely obsolete, the skills of general thinking and creativity must not become that. This couldn't be more specious. Meme thinking like this, repeating something you've heard as reflex without regard to whether it fits a situation, is the exact kind of unoriginality we can't allow to become the default mode of thinking.

I find the AI can make me more creative. I don't have to waste mental energy on boilerplate or straightforward stuff that would take me typing through some event processing loop etc. I can extract out and reuse components easier and focus on big picture design. Or build more bespoke admin tools that I wouldn't have wanted to waste time building some JS stuff before.
Post reply on HN