Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Superintelligence Labs' first paper is about RAG

paddedinputs.substack.com

Post reply on HN
Print view View original

211–220 of 283 posts

Re: Meta Superintelligence Labs' first paper is about RAG

#211

Post by gessha »

pityJuke wrote: post #171
ipsum2 wrote: post #17

This has nothing to do with superintelligence, it's just the people that were working on the paper prior to the re-org happened to publish after the name change. Though it is notable that contrary to many (on HN and Twitter) that Meta would stop publishing papers and be like other AI labs (e.g. OpenAI). They're continued their rapid pace of releasing papers AND open source models.

What model(s) have Meta released since the Lab re-org? Also, that wasn't based on purely hearsay, Zuck explicitly said: > We believe the benefits of superintelligence should be shared with the world as broadly as possible. That said, superintelligence will raise novel safety concerns. We'll need to be rigorous about mitigating these risks and careful about what we choose to open source. Still, we believe that buildin…

You still believe anything that comes out of his mouth?

Re: Meta Superintelligence Labs' first paper is about RAG

#213

Post by phkahler »

HPsquared wrote: post #162
ekianjo wrote: post #136

Earlier quoted context omitted.

Open weights models, not open source. And even their weights are under a specific license not as permissive as apache 2.

This is the right terminology. Model weights are literally compiled binary data; they are the output of an algorithm run on a bunch of source data. That training dataset is the "source" of the model. Training data (or the scripts used to generate it) is human-readable and modifiable, like source code. Binary weights are not.

Binary weights can still be "edited" with additional training.

Re: Meta Superintelligence Labs' first paper is about RAG

#215

Post by seanmcau »

bobxmax wrote: post #173
tsunamifury wrote: post #45

Earlier quoted context omitted.

Why in the world would he want Murati? She has absolutely no technical chops and was not functionally CTO of OpenAI.

What technical chops does Sam Altman have?

He started coding at age 8

Re: Meta Superintelligence Labs' first paper is about RAG

#216

Post by lukev »

puttycat wrote: post #41

Seems very incremental and very far from the pompous 'superintelligence' goal.

Think about it this way; they are encoding whole "thoughts" or "ideas" as single tokens.

It's effectively a multimodal model, which handles "concept" tokens alongside "language" tokens and "image" tokens.

A really big conceptual step, actually, IMO.

Re: Meta Superintelligence Labs' first paper is about RAG

#217

Post by senderista »

jongjong wrote: post #130
senderista wrote: post #8

Earlier quoted context omitted.

Yup, when I was at MSFT 20 years ago they were already productizing vector embedding of documents and queries (LSI).

Interesting. Makes one think.

To be clear, LSA[1] is simply applied linear algebra, not ML. I'm sure learned embeddings outperform the simple SVD[2] used in LSA.

[1] https://en.wikipedia.org/wiki/Latent_semantic_analysis

[2] https://en.wikipedia.org/wiki/Singular_value_decomposition

Re: Meta Superintelligence Labs' first paper is about RAG

#218

Post by godelski »

AnthonyMouse wrote: post #125
godelski wrote: post #97

Earlier quoted context omitted.

> an objective metric I'd like to push back on this a little, because I think it's important to understanding why Goodhart's Law shows up so frequently. * There are no /objective/ metrics* , only proxies. You can't measure a meter directly, you have to use a proxy like a tape measure. Similarly you can't measure time directly, you have to use a stop watch. In a normal conversation I wouldn't be nitpicking like this b…

This isn't what Goodhart's law is about. Sure, if you get a ruler from the store it might be off by a fraction of a percent in a way that usually doesn't matter and occasionally does, but even if you could measure distance exactly that doesn't get you out of it. Because what Goodhart's law is really about is bureaucratic cleavage. People care about lots of diverging and overlapping things, but bureaucratic rules don'…

I agree with you, in a way.

I've just taken another step to understand the philosophy of those bureaucrats. Clearly they have some logic, right? So we have to understand why they think they can organize and regulate from the spreadsheet. Ultimately it comes down to a belief that the measurements (or numbers) are "good enough" and that they have a good understanding of how to interpret them. Which with many bureaucracies that is the belief that no interpretation is needed. But we also see that behavior with armchair experts who try to use data to evidence their conclusion rather than interpret data and conclude from that interpretation.

Goodhart had focused on the incentive structure of the rule, but that does not tell us how this all happens and why the rule is so persistent. I think you're absolutely right that there is a problem with agents, and it's no surprise that when many introduce the concept of "reward hacking" that they reference Goodhart's Law. Yes, humans can typically see beyond the metric and infer the intended outcome, but ignore this because they don't care and so fixate on the measurement because that gives them the reward. Bureaucracies no doubt amplify this behavior as they are well known to be soul crushing.

But we should also be asking ourselves if the same effect can apply in settings where we have the best of intentions and all the agents are acting in good faith and trying to interpret the measure instead of just game it. The answer is yes. Idk, call it Godelski's Corollary if you want (I wouldn't), but it this relates to Goodhart's Law at a fundamental level. You can still have metric hacking even when agents aren't aware or even intending to do so. Bureaucracy is not required.

Re: Meta Superintelligence Labs' first paper is about RAG

#219

Post by godelski »

Eisenstein wrote: post #155
ivanbelenky wrote: post #127

Earlier quoted context omitted.

Thanks for sharing. I did not know this law existed and had a name. I know nothing about nothing but it appears to be the case that the interpretation of metrics for policies assume implicitly the "shape" of the domain. E.g. in RL for games we see a bunch of outlier behavior for policies just gaming the signal. There seems to be 2 types - Specification failure: signal is bad-ish, a completely broken behavior --> loca…

I view Goodhart's law more as a lesson for why we can never achieve a goal by offering specific incentives if we are measuring success by the outcome of the incentives and not by the achievement of the goal. This is of course inevitable if the goal cannot be directly measured but is composed of many constantly moving variables such as education or public health. This doesn't mean we shouldn't bother having such goals…

  > This is of course inevitable if the goal cannot be directly measured
It's worth noting that no goal can be directly measured[0].

I agree with you, this doesn't mean we shouldn't bother with goals. They are fantastic tools. But they are guides. The better aligned our proxy measurement is with the intended measurement then the less we have to interpret our results. We have to think less, spending less energy. But even poorly defined goals can be helpful, as they get refined as we progress in them. We've all done this since we were kids and we do this to this day. All long term goals are updated as we progress in them. It's not like we just state a goal and then hop on the railroad to success.

It's like writing tests for code. Tests don't prove that your code is bug free (can't write a test for a bug you don't know about: unknown unknown). But tests are still helpful because they help evidence the code is bug free and constrain the domain in which bugs can live. It's also why TDD is naive, because tests aren't proof and you have to continue to think beyond the tests.

[0] https://news.ycombinator.com/item?id=45555551

Re: Meta Superintelligence Labs' first paper is about RAG

#220

Post by godelski »

vasco wrote: post #132
exe34 wrote: post #116

Earlier quoted context omitted.

it doesn't break down - see comments about rules above. it was the perfect example to prove yourself wrong.

I disagree with all of those examples, they are misunderstanding what it means for the metric to break down in the context of the law, but alas. "If you run a different race" lol.

  > in the context of the law
That's the key part. The metric has context, right?

And that's where Goodhart's "Law" comes in. A metric has no meaning without context. This is why metrics need to be interpreted. They need to be evaluated in context. Sometimes this context is explicit but other times it is implicit. Often people will hack the metric as the implicit rule is not explicit and well that's usually a quick way to make those rules explicit.

Here's another way to think about it: no rule can be so perfectly written that it has no exceptions.

Post reply on HN