Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Superintelligence Labs' first paper is about RAG

paddedinputs.substack.com

Post reply on HN
Print view View original

11–20 of 283 posts

Re: Meta Superintelligence Labs' first paper is about RAG

#11

Post by liampulles »

jongjong wrote: post #4

Interesting. All developers I know who tinkered around with embeddings and vector similarity scoring were instantly hooked. The efficiency of computing the embeddings once and then reusing as many times as needed, comparing the vectors with a cheap IMO vector embedding is the most important innovation in computing of the last decade. There's something magical about it. These people deserve some kind of prize. The ide…

If you take the embedding for king, subtract the embedding for male, add the embedding for female, and lookup the closest embedding you get queen.

The fact that dot product addition can encode the concept of royalty and gender (among all other sorts) is kind of magic to me.

Re: Meta Superintelligence Labs' first paper is about RAG

#12

Post by naasking »

> the core insight here is actually: if embeddings are generated by layers within the LLM, it makes no sense to convert them back to natural language, just for another LLM to compress those tokens back to embeddings.

Doesn't this tie the two layers together in a way that they can't evolve separately?

Re: Meta Superintelligence Labs' first paper is about RAG

#14

Post by fishmicrowaver »

xvector wrote: post #13

Working in big tech it's pretty wild to see how integral AI has become to our work internally, vs the public perception of it. People are NOT prepared.

Not prepared for what? Seems like the rest of the world is desperate to be shown the way to unlock something of value?

Re: Meta Superintelligence Labs' first paper is about RAG

#15

Post by godelski »

It's kinda funny, Meta has long had some of the best in the field, but left them untapped. I really think if they just took a step back and stop being so metric focused and let their people freely explore then they'd be winning the AI race. But with this new team, I feel like meta mostly hired the people who are really good at gaming the system. The people that care more about the money than the research.

A bit of this is true at every major lab. There's tons of untapped potential. But these organizations are very risk adverse. I mean why not continue with the strategy that got us to the point we're at in the first place. Labs used to hire researchers and give them a lot of free reign. But those times ended and AI progress also slowed down. Maybe if you want to get ahead you gotta stop thinking like everyone else

Well meta... you can "hold me hostage" for a lot cheaper than those guys. I'm sure this is true for hundreds of passionate ML researchers. I'd take a huge pay cut to have autonomy and resources. I know for a fact there's many working at Mets right now that would do the same. Do maybe if you're going to throw money at the problem, diversify a bit and look back at what made SV what it is today and what made AI take leaps forward

Re: Meta Superintelligence Labs' first paper is about RAG

#16

Post by terminalshort »

xvector wrote: post #13

Working in big tech it's pretty wild to see how integral AI has become to our work internally, vs the public perception of it. People are NOT prepared.

1. Hyperbolic statement about LLM capabilities with no concrete examples

2. Wild claim that the companies that sell LLMs are actually downplaying their capabilities instead of hyping them

Re: Meta Superintelligence Labs' first paper is about RAG

#17

Post by ipsum2 »

This has nothing to do with superintelligence, it's just the people that were working on the paper prior to the re-org happened to publish after the name change.

Though it is notable that contrary to many (on HN and Twitter) that Meta would stop publishing papers and be like other AI labs (e.g. OpenAI). They're continued their rapid pace of releasing papers AND open source models.

Re: Meta Superintelligence Labs' first paper is about RAG

#18

Post by Workaccount2 »

fishmicrowaver wrote: post #14
xvector wrote: post #13

Working in big tech it's pretty wild to see how integral AI has become to our work internally, vs the public perception of it. People are NOT prepared.

Not prepared for what? Seems like the rest of the world is desperate to be shown the way to unlock something of value?

I think at this point it's software devs looking for the value unlock.

Non-software devs are actually making functional programs for themselves for the first time ever. The value is crazy.

Re: Meta Superintelligence Labs' first paper is about RAG

#19

Post by ceejayoz »

Workaccount2 wrote: post #18
fishmicrowaver wrote: post #14

Earlier quoted context omitted.

Not prepared for what? Seems like the rest of the world is desperate to be shown the way to unlock something of value?

I think at this point it's software devs looking for the value unlock. Non-software devs are actually making functional programs for themselves for the first time ever. The value is crazy.

It’s not the first time ever. People did the same with Access and HyperCard in the 90s.
Post reply on HN