Seems very incremental and very far from the pompous 'superintelligence' goal.
Meta Superintelligence Labs' first paper is about RAG
41–50 of 283 posts
Re: Meta Superintelligence Labs' first paper is about RAG
#42Interesting. All developers I know who tinkered around with embeddings and vector similarity scoring were instantly hooked. The efficiency of computing the embeddings once and then reusing as many times as needed, comparing the vectors with a cheap IMO vector embedding is the most important innovation in computing of the last decade. There's something magical about it. These people deserve some kind of prize. The ide…
If you take the embedding for king, subtract the embedding for male, add the embedding for female, and lookup the closest embedding you get queen. The fact that dot product addition can encode the concept of royalty and gender (among all other sorts) is kind of magic to me.
This was actually shown to not really work in practice.
Re: Meta Superintelligence Labs' first paper is about RAG
#43Did a "superintelligence" lab publish a superintelligence related paper with no results for intelligence? What measured improvements did this proposal make in their LLM's intelligence?
Re: Meta Superintelligence Labs' first paper is about RAG
#44It's kinda funny, Meta has long had some of the best in the field, but left them untapped. I really think if they just took a step back and stop being so metric focused and let their people freely explore then they'd be winning the AI race. But with this new team, I feel like meta mostly hired the people who are really good at gaming the system. The people that care more about the money than the research. A bit of th…
I thought Alex Wang was a very curious choice. There are so many foundational AI labs with interesting CEOs... I get that Wang is remarkable in his own right, but he basically just built MTurk and timed the bubble. Doesn't really scream CEO of AGI to me.
Alexandr Wang is not interesting and a few steps short of a fraud that Mark had to bail out because he was so co invested.
Shareholders should be livid if they knew a single thing about what was going on.
Re: Meta Superintelligence Labs' first paper is about RAG
#45Earlier quoted context omitted.
I thought Alex Wang was a very curious choice. There are so many foundational AI labs with interesting CEOs... I get that Wang is remarkable in his own right, but he basically just built MTurk and timed the bubble. Doesn't really scream CEO of AGI to me.
The reportings at the time said that he was Mark’s 5th choice or similar. It is fairly clear he would prefer Ilya, Murati, Mark Chen, and perhaps others, but they said no, and Alex Wang was the first one to say yes.
Why in the world would he want Murati? She has absolutely no technical chops and was not functionally CTO of OpenAI.
Re: Meta Superintelligence Labs' first paper is about RAG
#46Earlier quoted context omitted.
The reportings at the time said that he was Mark’s 5th choice or similar. It is fairly clear he would prefer Ilya, Murati, Mark Chen, and perhaps others, but they said no, and Alex Wang was the first one to say yes.
Why in the world would he want Murati? She has absolutely no technical chops and was not functionally CTO of OpenAI.
Because she was CTO of OpenAI.
Re: Meta Superintelligence Labs' first paper is about RAG
#47Earlier quoted context omitted.
The reportings at the time said that he was Mark’s 5th choice or similar. It is fairly clear he would prefer Ilya, Murati, Mark Chen, and perhaps others, but they said no, and Alex Wang was the first one to say yes.
Why in the world would he want Murati? She has absolutely no technical chops and was not functionally CTO of OpenAI.
> was not functionally CTO of OpenAI.
Why do you say that?
Re: Meta Superintelligence Labs' first paper is about RAG
#48Re: Meta Superintelligence Labs' first paper is about RAG
#49Earlier quoted context omitted.
I thought Alex Wang was a very curious choice. There are so many foundational AI labs with interesting CEOs... I get that Wang is remarkable in his own right, but he basically just built MTurk and timed the bubble. Doesn't really scream CEO of AGI to me.
The reportings at the time said that he was Mark’s 5th choice or similar. It is fairly clear he would prefer Ilya, Murati, Mark Chen, and perhaps others, but they said no, and Alex Wang was the first one to say yes.
The self-supervised mesa-optimizer strikes again
Re: Meta Superintelligence Labs' first paper is about RAG
#50Earlier quoted context omitted.
I thought Alex Wang was a very curious choice. There are so many foundational AI labs with interesting CEOs... I get that Wang is remarkable in his own right, but he basically just built MTurk and timed the bubble. Doesn't really scream CEO of AGI to me.
A lot of people also don't know that many of the well known papers are just variations on small time papers with a fuck ton more compute thrown at the problem. Probably the strongest feature that correlates to successful researcher is compute. Many have taken this to claim that the GPU poor can't contribute but that ignores so many other valid explanations... and we wonder why innovation has slowed... It's also weird…
Frankly this is the reason why Im not convinced the current movement of LLMs will yield anything close to the dream.
The right people to deliver immense progress dont exist right now.