Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How LLMs work

0xkato.xyz

Post reply on HN
Print view View original

61–70 of 293 posts

Re: How LLMs work

#61

Post by mejutoco »

ai_slop_hater wrote: post #51
baq wrote: post #41

Earlier quoted context omitted.

We don’t treat children like they’re stupid, we treat children like they’re children. A stupid adult is treated very differently than any child. Adults are expected to have their world models approximately correct in terms of physical environment so they won’t accidentally kill themselves by falling off a cliff; then there are the social norms which adults are expected to conform to so everyone is kinda predictable t…

You may have been raised properly since you don’t get what I mean. I really envy kids with “Chinese parents” that had them learn math early on and not some bullshit like that if you put your tooth under your pillow, then a tooth fairy will come.

I think those 2 are orthogonal. Math still works with Santa or the tooth fairy.

Re: How LLMs work

#62

Post by rfv6723 »

ctolsen wrote: post #38
10GBps wrote: post #15

Earlier quoted context omitted.

Yep. It's nearly identical to the neural nets we were using in the 90s. Back then even a supercomputer wasn't big enough or fast enough to do what we do today. I have to wonder though. Is this all a human brain is? A similar thing to an LLM just scaled exponentially larger. I mean a brain is not just neurons with simple connections to each other. The neurons, axons, dendrites, , etc in a brain are all holding and pro…

No, it’s definitely not what a human brain is. That makes very little sense. The ways we interact with language (and thus conceptual memory) is completely and fundamentally different.

Is it different though?

If we look beyond written languages which are late inventions of human civilization, oral languages are continuous and build with blocks not words.

Chomskyan school misled the entire field of linguistics for decades by ignoring spoken languages.

Re: How LLMs work

#63

Post by ai_slop_hater »

pmg101 wrote: post #57
ai_slop_hater wrote: post #36

Earlier quoted context omitted.

Human brain capabilities are truly amazing, imagine if people didn’t treat their children as if they are stupid and didn’t constantly lie to them, because kids are stupid right, they wouldn’t understand. What heights could be reached.

Because god forbid that childhood, the one time in your life when you don't have any responsibilities, should be fun.

Waste 22 years of life without learning anything and then slave away at a 9-5 job you hate. Brilliant strategy. At least you had “fun”. Then blame billionaires or something.

Re: How LLMs work

#64

Post by barrenko »

sharma-arjun wrote: post #58
wuschel wrote: post #33

Earlier quoted context omitted.

Could you perhaps cite the core papers for LLMs beyond „Attention is all you need“?

Not a core paper, but I found Formal Algorithms for Transformers [1] (a Google paper from 2022) to have a great pedagogical style. [1] https://arxiv.org/abs/2207.09238

I'll add in here https://web.stanford.edu/~jurafsky/slp3/, "Speech and Language Processing", with chapters that deal specifically with LLMs and transformers.

Re: How LLMs work

#65

Post by lionkor »

It sucks that this article is clearly LLM edited, with common phrases like "same shape as", "the intuition: ", and the "tiny explainer" which clearly generalized from a prompt accidentally.

Good article, but when sharing it I will have to preface "yes it's slop, but it's a good explanation".

Absolutely embarrassing that the author didn't catch that these LLM-isms are a (and here I'll use one) bad signal.

In fact, I would go so far as to say that publishing in this style stems from a lack of reading experience and writing experience, which does not bode well for someone pretending to be an expert. I gave this article to someone highly intelligent who doesn't know the first thing about how LLMs work internally, and she immediately called out that it reads like AI text.

Re: How LLMs work

#66

Post by beezlewax »

ai_slop_hater wrote: post #51
baq wrote: post #41

Earlier quoted context omitted.

We don’t treat children like they’re stupid, we treat children like they’re children. A stupid adult is treated very differently than any child. Adults are expected to have their world models approximately correct in terms of physical environment so they won’t accidentally kill themselves by falling off a cliff; then there are the social norms which adults are expected to conform to so everyone is kinda predictable t…

You may have been raised properly since you don’t get what I mean. I really envy kids with “Chinese parents” that had them learn math early on and not some bullshit like that if you put your tooth under your pillow, then a tooth fairy will come.

It is possible to have learned both things you know.

Re: How LLMs work

#67

Post by throw310822 »

otabdeveloper4 wrote: post #43
jumploops wrote: post #22

Earlier quoted context omitted.

Those are all just optimizations. We still don’t really know why they work, we just know how to build them.

We do know how they work. They predict the next statistically most likely token. The "bitter lesson" is that fake-it-till-you-make-it is a valid way of doing knowledge work. (Or not make it, then people will just claim you're holding the LLM wrong and it's not the AI's fault.)

> statistically most likely token.

Statistically most likely in what context, given which preconditions? Because each prompt sequence is unique so the probability of any token following it is unknown.

Re: How LLMs work

#68

Post by vocram »

Saying an article is of inferior quality just because editing was AI-assisted is like saying a book is lower quality just because it was printed rather than written by hand

Re: How LLMs work

#70

Post by ai_slop_hater »

mejutoco wrote: post #61
ai_slop_hater wrote: post #51

Earlier quoted context omitted.

You may have been raised properly since you don’t get what I mean. I really envy kids with “Chinese parents” that had them learn math early on and not some bullshit like that if you put your tooth under your pillow, then a tooth fairy will come.

I think those 2 are orthogonal. Math still works with Santa or the tooth fairy.

Maybe math works but critical thinking doesn’t. There are people who have lived for many decades without ever questioning insane b.s. they were taught as kids.
Post reply on HN