Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Superintelligence Labs' first paper is about RAG

paddedinputs.substack.com

Post reply on HN
Print view View original

261–270 of 283 posts

Re: Meta Superintelligence Labs' first paper is about RAG

#261

Post by godelski »

bwfan123 wrote: post #189
hamasho wrote: post #62

Earlier quoted context omitted.

My theory is that as more people compete, the top candidates become those who are best at gaming the system rather than actually being the best. Someone has probably studied this. My only evidence is job applications for GAFAM and Tinder tho.

I would categorize people into 2 broad extremes. 1) those that care two hoots about what others or the system expects of them and in that sense are authentic and 2) those that only care about what others or the system expects of them, and in that sense are not authentic. There is a spectrum in there.

If you haven't heard this already, you might be interested in Pournelle's Iron Law of Bureaucracy

https://jerrypournelle.com/reports/jerryp/iron.html

Re: Meta Superintelligence Labs' first paper is about RAG

#263

Post by danielmarkbruce »

godelski wrote: post #258
danielmarkbruce wrote: post #257

Earlier quoted context omitted.

It's just plain old addition. There is nothing fancy about the operation. The fancy part is training a model such that it would produce vector representations of words which had this property of conceptually making sense. If someone says: "conceptually, what is king - man + woman". One might reasonably say "queen". This isn't some well defined math thing, just sort of a common sense thing. Now, imagine you have a fun…

> It's just plain old addition I'm sorry, but I think you are overestimating your knowledge. Have you gone through abstract algebra? Are you familiar with monoids, groups, rings, fields, algebras, and so on? Because it seems you aren't aware that these structures exist and area critical part of mathematics. It's probably why you're not understanding the conversation. @yellocake seems to understand that "addition" doe…

Maybe spend more time reading a response than writing. Yellowcake doesn't know what you are talking about either (note the "pulling teeth" comment).

The examples you gave are a result of the embedding model in question not producing vectors which would map to most peoples conceptual view of the world. Go through that website you quote from and see for yourself - it's just element wise addition.

The examples I gave are entirely made up, 2 dimensional vectors to explain what plain old addition means (ie, plain old "add the vectors element wise") in the context of embedding models. And yes, it's the same as, because I defined it that way. Your website uses 300 dimensions, not 2.

As I mentioned, not all embedding models work the same way (or, as you've said, "this doesn't generalize"). They get trained differently, on different data. The word "similar" is used very loosely.

You even directly quote me and don't seem to be able to read the quote. The word "could" is there. You could end up with a model which had these nice properties.

The entire point of my post was to highlight that yellowcake's confusion arises because he assumes it's an esoteric definition of addition that results in your examples, when it's not that.

Re: Meta Superintelligence Labs' first paper is about RAG

#264

Post by godelski »

danielmarkbruce wrote: post #263
godelski wrote: post #258

Earlier quoted context omitted.

> It's just plain old addition I'm sorry, but I think you are overestimating your knowledge. Have you gone through abstract algebra? Are you familiar with monoids, groups, rings, fields, algebras, and so on? Because it seems you aren't aware that these structures exist and area critical part of mathematics. It's probably why you're not understanding the conversation. @yellocake seems to understand that "addition" doe…

Maybe spend more time reading a response than writing. Yellowcake doesn't know what you are talking about either (note the "pulling teeth" comment). The examples you gave are a result of the embedding model in question not producing vectors which would map to most peoples conceptual view of the world. Go through that website you quote from and see for yourself - it's just element wise addition. The examples I gave ar…

  > Maybe spend more time reading a response than writing.
Quite ironic considering

  > Yellowcake doesn't know what you are talking about either
I actually said

  >> @yellocake seems to understand that "addition" doesn't mean 'addition'
Which is entirely based off of

  >>>>>> Presumably you're overloading the addition symbol
I didn't assume their knowledge, they straight up told me and I updated my understanding based on that. That's how conversations work. And the fact that they understand operator overloading doesn't mean they understand more either. Do they understand monoids, fields, groups, and rings? Who knows? We'll have to let yellowcake tell us.

Regardless, what you claim I assumed about yellowcake's knowledge is quite different than what I actually said. So maybe take your own advice.

I write a lot because, unlike you, I understand these things are complex. Were it simpler, I would not need as many words.

Re: Meta Superintelligence Labs' first paper is about RAG

#265

Post by danielmarkbruce »

godelski wrote: post #264
danielmarkbruce wrote: post #263

Earlier quoted context omitted.

Maybe spend more time reading a response than writing. Yellowcake doesn't know what you are talking about either (note the "pulling teeth" comment). The examples you gave are a result of the embedding model in question not producing vectors which would map to most peoples conceptual view of the world. Go through that website you quote from and see for yourself - it's just element wise addition. The examples I gave ar…

> Maybe spend more time reading a response than writing. Quite ironic considering > Yellowcake doesn't know what you are talking about either I actually said >> @yellocake seems to understand that "addition" doesn't mean 'addition' Which is entirely based off of >>>>>> Presumably you're overloading the addition symbol I didn't assume their knowledge, they straight up told me and I updated my understanding based on th…

Yeah except addition does mean addition in this case - ask anyone what plain old addition means for a vector, and they'll tell you element wise addition. The website you quoted is for a simple example using element wise addition and you made it sound as complex as possible because you are desperate to sound smart.

Re: Meta Superintelligence Labs' first paper is about RAG

#266

Post by danielmarkbruce »

BoorishBears wrote: post #214
danielmarkbruce wrote: post #48

Earlier quoted context omitted.

That was sarcasm. He's not lying.

Didn't read any sarcasm in what he said?

Sorry, I meant my comment was sarcasm. I was being sarcastic. The original comment was sincere, I'm quite certain. And, they are right - there are some companies that really are getting a lot of value out of LLMs already. I'd guess that the more folks who actually understand how LLMs work, the more a company can do. There just isn't a neat abstraction layer to be had, so folks who don't have a detailed mental model get caught up applying them poorly or to the wrong things.

Re: Meta Superintelligence Labs' first paper is about RAG

#267

Post by fishmicrowaver »

Workaccount2 wrote: post #196
fishmicrowaver wrote: post #23

Earlier quoted context omitted.

Sure but I'm the real world do you think businesses are going to deploy piles of code into production generated this way? No, non technical people will continue to whip up MS PowerApps. AI generated code has no value to many businesses.

You don't need production level code to make your life easier. You're missing the forest for the trees. Most people can't even make a block diagram, but they can explain what they have and what they want to do with it.

I think the market reveals itself. Perhaps you're right, but it's been years, and where's the value? No offense, and it might seem cool to build an app, but that's been possible for decades.

Re: Meta Superintelligence Labs' first paper is about RAG

#268

Post by godelski »

danielmarkbruce wrote: post #265
godelski wrote: post #264

Earlier quoted context omitted.

> Maybe spend more time reading a response than writing. Quite ironic considering > Yellowcake doesn't know what you are talking about either I actually said >> @yellocake seems to understand that "addition" doesn't mean 'addition' Which is entirely based off of >>>>>> Presumably you're overloading the addition symbol I didn't assume their knowledge, they straight up told me and I updated my understanding based on th…

Yeah except addition does mean addition in this case - ask anyone what plain old addition means for a vector, and they'll tell you element wise addition. The website you quoted is for a simple example using element wise addition and you made it sound as complex as possible because you are desperate to sound smart.

  > you are desperate to sound smart.
Because I don't think 1-1+1=7?

Whatever you say man

Re: Meta Superintelligence Labs' first paper is about RAG

#269

Post by smokel »

danielmarkbruce wrote: post #246
smokel wrote: post #134

Earlier quoted context omitted.

> I certainly didn't judge them because they are just playing the game. Please do judge them for being parasitical. They might seem successful by certain measures, like the amount of money they make, but I for one simply dislike it when people only think about themselves. As a society, we should be more cautious about narcissism and similar behaviors. Also, in the long run, this kind of behaviour makes them an annoyi…

There is an implication that passionate weirdos are good by nature. You either add value in the world or you don't. A passionate, strange actor or musician who continues trying to "make it" who isn't good enough to be entertaining is a parasite and/or narcissist. A plumber who is doing the job purely for money is a value add (assuming they aren't ripping people off) - and they are playing the game - the money for wor…

I'm not a plumber, but I think the analogy would require the plumber to be strategical about every job they take on. Within a few months, our plumber would only be plumbing for millionaires, installing golden faucets at extreme price points. I would then stop befriending said plumber.

Re: Meta Superintelligence Labs' first paper is about RAG

#270

Post by danielmarkbruce »

godelski wrote: post #268
danielmarkbruce wrote: post #265

Earlier quoted context omitted.

Yeah except addition does mean addition in this case - ask anyone what plain old addition means for a vector, and they'll tell you element wise addition. The website you quoted is for a simple example using element wise addition and you made it sound as complex as possible because you are desperate to sound smart.

> you are desperate to sound smart. Because I don't think 1-1+1=7? Whatever you say man

You really don't understand that the illogical sounding results from that website are due to the vectors themselves huh. It has zero to do with the definition of +.
Post reply on HN